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PREFACE 


This  study  was  performed  by  the  Institute  for  Defense  Analyses  (IDA)  under  Task 
Order  T-L6-361  (Identifying  Personnel  Tradeoffs)  sponsored  by  the  Office  of  the  Assistant 
Secretary  of  Defense  (Force  Management  and  Personnel)  under  contract  MDA  903  84  C 
0031.  Its  main  focus  is  identification  of  sources  of  information  on  the  performance  of  units 
and  individuals  in  the  military.  The  existence  of  such  data  permits  the  development  of 
quantitative  relationships  between  policies  relating  to  manpower,  personnel  and  training 
(MPT)  and  performance  in  the  military.  These  relationships  could  be  used  to  improve  the 
way  in  which  MPT  policies  are  determined  and  evaluated.  They  could  help  military 
manpower  planners  buy  a  more  effective  defense  for  the  money. 

The  authors  are  grateful  for  the  many  helpful  suggestions  received  from  Dr. 
Deborah  Clay-Mendez,  Dr.  David  R.  Graham,  Dr.  James  S.  Thomason  and  Richard  S. 
Gibson.  The  assistance  of  Ms.  Eileen  M.  Doherty  and  Ms.  Crystal  A.  Moore  is  also 
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I.  INTRODUCTION 


A.  BACKGROUND 

This  paper  presents  the  results  of  research  conducted  in  identifying  existing  data  on 
personnel  characteristics,  training  and,  performance  in  the  military  Services.  The  end 
purpose  is  to  link  the  characteristics  of  unit  personnel  with  unit  performance.  Identifying 
these  links  would  serve  at  least  two  critical  functions:  assessing  manpower,  personnel  and 
training  (MPT)  policy  options,  and  deriving  measures  of  manpower  capability  from 
information  on  operational  performance. 

A  sample  of  data  bases  covering  the  performance  of  military  units  and  personnel,  as 
well  as  the  characteristics  and  training  of  people  in  the  military  are,  categorized  and 
analyzed  in  the  paper.  There  are  at  least  four  reasons  why  we  should  know  how  the 
individual  and  aggregate  characteristics  of  military  personnel  relate  to  their  collective 
performance  as  a  unit  These  are  to  be  able  to: 

•  Decide  who  to  enlist  and  retain,  and  in  what  numbers  to  enlist  and  retain  them; 

•  Disign  better  pay  and  retirement  policies  to  c  ompensate  them; 

•  Design  better  personnel  management  (e.g.,  rotation  and  training)  policies;  and 

•  Develop  aggregate  indicators  of  the  quality  of  military  personnel. 

Military  planners  and  policy  makers  should  be  able  to  assess  the  results  (outputs)  of 
their  plans  and  policies.  They  must  also  be  able  to  demonstrate  to  the  Congress  that  the 
output  justifies  the  expenditure  of  resources.  It  is  not  enough  to  be  able  to  show  that  more 
resources  of  a  given  kind  produce  more  capability;  good  policy  development  requires 
assessment  of  how  much  capability  is  gained  through  alternative  uses  of  the  resources. 

Much  of  the  past  analytic  work  in  the  manpower  policy  area  has  been  concentrated 
on  how  to  fill  stated  manpower  requirements.  This  work  needs  to  be  continued,  since  DoD 
certainly  should  be  concerned  with  efficiently  getting  the  necessary  people.  However, 
statements  of  requirements  often  have  been  based  on  implicit  and  unverified  assumptions 
about  the  required  mixes  of  age,  intelligence,  education  and  experience.  The  requirements 
are  derived  with  little  explicit  reference  to  how  well  other  mixes  might  perform  or  to  how 


yi 


t 


•I 


E* 


f 


S 


v. 


•5: 

y* 


r 

re 


much  they  might  cost  Compensation  and  retirement  policies  are  then  designed  to  fill  these 
requirements.  For  example,  little  attention  is  given  to  the  possibility  that  senior  people 
contribute  so  much  in  some  settings  that  money  could  be  saved  and  military  capability 
increased  by  paying  these  people  more  and  retaining  more  of  them,  while  bringing  ir.  fewer 
recruits  and  reducing  training  costs.  Under  the  current  system,  there  is  a  tendency  to  focus 
on  the  fact  that  this  action  would  result  in  overfilling  requirements  for  senior  personnel, 
leading  to  consideration  of  forcing  some  of  them  to  leave  in  order  to  maintain  a  viable 
promotion  plan. 

At  the  same  time,  the  retirement  system  has  ’  'n  reasonably  efficient  in  achieving 
the  level  of  post-20-year  personnel  implied  by  the  Services'  statements  of  requirements. 
However,  an  examination  of  units  with  more  men  of  "post-retirement"  age  might  show  this 
level  to  be  based  on  faulty  assumptions.  A  performance-oriented  approach  to  requirements 
might  suggest  further  modifications  to  the  retirement  system  that  would  allow  it  to  serve 
better  the  real  needs  of  the  Services. 


Another  consideration  is  training.  Considerable  effort  and  expense  go  into  the 
design  and  execution  of  training  programs.  Individual  training  programs  are  usually 
evaluated  on  the  basis  of  individual  test  results,  with  some  feedback  from  the  field.  Further 
follow-up  that  examines  the  performance  of  units,  while  controlling  for  specific  training 
and  other  personnel  characteristics,  could  give  vital  clues  to  people  who  design  training 
courses  and  programs.  Both  schoolhouse  and  unit  training  would  benefit  from  such 
examination. 


Tying  MPT  policies  to  performance  would  not  only  lead  to  better  MPT  policies,  but 
would  help  to  justify  them.  When  manpower,  personnel  and  training  resources  can  be 
linked  to  warfighting  capability,  their  need  and  cost  can  be  more  easily  compared  with 
those  of  additional  forces  and  modernization.  This  in  turn  should  lead  to  more  efficient  and 
effective  negotiations  for  budget  dollars. 

Building  such  links  requires  that  three  conditions  hold: 

•  that  data  reflecting  military  performance  exist; 

•  that  such  data  be  available  for  research  purposes;  and 

«  that  analyses  using  these  data  to  tie  MPT  policies  to  performance  be 
successfully  pursued. 

This  paper  addresses  the  f? rst  two  conditions,  the  existence  and  availability  of  relevant 
information  on  military  performance. 
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Skepticism  about  the  existence  of  the  necessary  data  has  been  a  considerable 
stumbling  block  to  the  kind  of  research  under  discussion.  It  has  been  noted  that:  "So 
difficult  is  measuring  military  effectiveness  that  some  researchers  believe  it  is  futile  to  try."1 
The  goal  of  this  paper  is  to  show  that  these  researchers  are  wrong.  All  the  Services  go  to 
considerable  effort  to  develop  indicators  that  are  closely  related  to  military  effectiveness; 
these  indicators  are  generally  used  for  management  purposes  in  the  field.  While  many  of 
them  are  not  forwarded  to  higher  headquarters,  they  do  exist  Our  objective  is  to  list  and 
describe  a  sample  of  these  indicators:  who  uses  them,  how  they  are  developed,  what  data 
elements  they  are  developed  from,  what  form  they  are  kept  in,  and  where  they  are  kept. 
We  hope  to  help  dispel  the  belief  that  manpower  policies  cannot  be  developed  and  assessed 
in  terms  of  their  impact  on  the  performance  of  military  units. 

Building  the  capability  to  develop  and  assess  manpower  policies  in  this  way  will 
not  be  easy.  It  will  require  a  broad-based,  bottom-up  research  effort.  Relationships 
between  the  characteristics  of  people  and  collective  performance  are  likely  to  differ  by 
occupation  and  kind  of  equipment,  so  many  different  environments  will  have  to  be 
studied.2  An  extended  effort  will  be  needed,  promising  results  in  the  early  to  mid  1990s. 

B  .  PLAN  OF  THE  STUDY 

A  taxonomy  for  categorizing  indicators  of  military  effectiveness  is  developed  in 
Chapter  II.  The  paper  turns  to  its  main  work  in  Chapter  HI,  listing  and  discussing  such 
indicators.  This  chapter  also  includes  information  regarding  the  availability  of  data  on  the 
characteristics  and  training  of  personnel,  since  such  data  are  just  as  important  as  are 
effectiveness  measures  for  the  kind  of  research  we  contemplate. 

The  final  chapter  suggests  how  to  proceed  in  order  both  to  advance  the  long-term 
goal  of  linking  the  characteristics  and  training  of  personnel  to  the  performance  of  their  units 
in  a  wide  range  of  contexts,  and  to  gain  valuable  insights  in  the  shorter  term. 


1  Nicholas  Bond  Jr.,  17th  International  Symposium  on  Applied  Military  Psychology,  (London:  ONR, 
London  Branch,  U.S.  Department  of  the  Navy,  July  1 98 1). 

2  For  evidence  that  relationships  differ  in  this  way,  see  Stanley  A.  Horowitz  and  Allan  Sherman,  Crew 
Characteristics  and  Ship  Condition,  CNS  1090,  Center  for  Naval  Analyses,  March  1977. 


II.  FRAMEWORK  DEVELOPMENT 


As  we  are  focusing  on  the  determinants  of  military  performance,  our  consideration 
of  a  framework  for  analysis  begins  by  considering  the  prerequisites  for  unit  warfighting 
capability.3  Our  approach  is  to  start  with  the  result  and  move  one  step  at  a  time  to  the 
starting  point,  in  terms  of  the  flow  from  resources  to  capability.  At  each  step  the  question 
is  asked,  "What  do  we  need  to  get  that?'  By  the  end  of  the  process  the  categories  of 
resources  that  MPT  policies  are  concerned  with  are  reached  and  by  retracing  these  steps, 
the  links  that  must  be  forged  to  connect  MPT  policies  with  unit  effectiveness  are  seen. 
Also  evident  are  incremental  steps  that  can  be  taken  to  make  MPT  decisions  more  output- 
oriented,  even  if  the  most  difficult  links  remain  unforged. 

A.  THE  DETERMINANTS  OF  UNIT  CAPABILITY 

Unit  capability  depends  upon  the  resources  made  available  and  what  is  done  with 
those  resources.  In  terms  of  accepted  military  definitions,  a  unit's  capability  is  determined 
by  its  designed  capability  and  its  readiness.  Readiness  is  the  ability  of  a  force,  unit, 
weapon  system  or  equipment  to  perform  the  mission  for  which  it  was  organized  or 
designed.  Since  MPT  policy  affects  defense  capability  largely  through  its  influence  on 
readiness,  the  discussion  will  be  concentrated  on  the  determinants  of  readiness. 

B .  CATEGORIZATION  OF  READINESS 

The  standard  treatment  of  readiness  breaks  overall  readiness  into  four  components: 
material,  personnel,  training,  and  supply  readiness  (see  Figure  2-1).  However,  this 
taxonomy  tends  to  mask  complex  interactions  among  the  components  of  readiness.  It 
perpetuates  the  incorrect  notion  that  these  are  four  parallel  components  that  do  not  affect 


3  For  a  detailed  discussion  of  a  framework  for  decision  making  which  includes  a  wider  spectrum  of  war¬ 
fighting  capability,  see  Stanley  A.  Horowitz,  Evaluating  Navy  Manpower .  Personnel  and  Training 
Policies  in  Terms  of  Performance,  Institute  for  Defense  Analyses,  IDA  Paper  P-1919,  March  1986. 


Figure  2-1.  The  Official  Vlaw  of  Raadineas-Four  Pillars 

each  other.  For  example,  the  concept  that  having  better  people  can  result  in  better  material 
readiness  is  not  formally  admitted  to  the  structure;  neither  is  the  possibility  considered  that 
unit  training  can  enhance  the  performance  of  individual  tasks.  The  absence  of  such  links 
severely  limits  the  value  of  this  structure  as  a  framework  to  guide  analysis. 

As  Figure  2-2  shows,  three  factors  must  be  present  for  a  unit  to  be  ready  for 
combat  Equipment  must  work  —  material  readiness  must  be  high.  The  crew  must  be 
proficient  in  using  the  equipment  in  combat  situations  -  training  readiness  must  be  high. 
Finally,  the  necessary  consumables  -  fuel,  ordnance  and  spare  parts  must  be  available  -- 
supply  readiness  must  be  high.  Personnel  readiness  -  the  ability  of  the  people  assigned  to 
a  unit  to  perform  their  individual  tasks  --  underpins  all  of  these  factors.  Effective  supply 
personnel  are  critical  to  supply  readiness  and  effective  maintenance  personnel  are  critical  to 
material  readiness.  Good  people  are  necessary  to  achieve  the  degree  of  training  readiness 
needed  for  operational  success. 
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Figure  2-2.  A  Modified  View  of  Readiness 


C.  PERSPECTIVES  ON  PERSONNEL  READINESS 

As  shown  in  Figure  2-3,  personnel  readiness  is  at  the  heart  of  a  complex  network 
of  interrelationships  that  underlies  unit  readiness.  Effective  personnel  see  to  it  that  the 
necessary  supplies  are  available.  Material  readiness  is  determined  by  equipment  reliability 
and  how  long  it  takes  to  repair  equipment  when  it  is  down.  Reliability  is  usually  taken  to 
depend  only  on  equipment  design  and  the  operating  environment  (including  the  tempo  of 
training-related  operations),  but  there  are  also  personnel  induced  failures.  These 
presumably  reflect  the  level  of  personnel  readiness  and  unit  training.  The  speed  of  repair  is 
dependent  upon  the  availability  of  spare  parts  as  well  as  the  availability  and  ability  of 
personnel  --  on  personnel  readiness.  Maintenance  is,  of  course,  a  team  effort.  The  speed 
of  repair  is  therefore  dependent  not  only  on  the  quality  and  quantity  of  repair  personnel,  but 
also  on  their  level  of  unit  training. 

The  level  of  unit  training  is  often  assumed  to  be  the  only  factor  underlying  training 
readiness  --  that  is,  the  ability  of  units  to  use  their  equipment  effectively  when  it  is 
working.  It  is  reasonable  to  believe,  however,  that  there  is  a  trade-off  between  personnel 
readiness  and  unit  training  in  the  v.iainment  of  training  readiness.  Thus,  it  should  be 
possible  to  achieve  a  given  level  of  operating  proficiency  with  fewer  team  skills  if 
personnel  readiness  --  the  quality  and  quantity  of  personnel  —  is  higher. 
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Figure  2-3.  The  Role  of  Personnel  and  Training  in  Producing  Unit  Readiness 


The  accomplishment  of  unit  training  can  hone  the  performance  of  individual  tasks  at 
the  same  time  it  improves  the  performance  of  team  tasks.  The  level  of  unit  training,  in  turn, 
depends  not  only  on  how  much  unit  training  is  done,  but  also  on  how  much  is  needed. 
This  is  a  function  of  personnel  turnover,  as  well  as  the  degree  to  which  skills  are 
perishable. 

Personnel  readiness  --  hew  well  people  can  do  their  individual  jobs  --  is  influenced 
by  the  characteristics  of  personnel  in  the  unit,  the  number  of  personnel,  grade  structure, 
unit  and  schoolhouse  training,  unit  leadership  and  motivation.  Personnel  characteristics 
include  what  individuals  entered  the  Service  with,  such  as  intelligence  and  education.  They 
also  include  individual  training,  military  experience  and  education  acquired  while  in  the 
Service. 


D.  DETERMINANTS  OF  PERSONNEL  CHARACTERISTICS 


Personnel  characteristics  inc  lude  the  measures  of  personnel  quality  most  often  used 
in  judging  the  success  of  MPT  policies:  intelligence  (mental  group  as  measured  by  entry 
test  scores),  education  level  (the  fraction  who  graduated  from  high  school)  and  military 
experience  (length  of  time  in  the  Service).  We  are  interested  in  expanding  this  list  of 
measures  of  personnel  quality  and  sharpening  their  definitions  in  order  to  get  greater  insight 
into  the  effect  of  MPT  policies  on  performance.  For  example,  time  in  service  is  one 
measure  commonly  used  for  military  experience.  Of  course,  time  in  service  is  only  a  proxy 
for  certain  work  experience  and  the  natural  maturing  process  which  takes  place  during  a 
person's  service.  More  useful  measures  of  experience  in  a  given  instance  might  include 
such  measures  as  flight  hours,  or  time  in  a  particular  job  or  type  of  unit.  A  complete 
treatment  of  how  personnel  affect  unit  performance  should  also  address  characteristics  such 
as  motivation  and  leadership,  which  are  determined  by  policies  that  are  both  internal  and 
external  to  the  unit 

These  characteristics  are  the  factors  we  want  to  use  to  quantify  personnel  quality. 
Some,  such  as  motivation  and  leadership,  are  very  difficult  to  measure.  Data  on  others  are 
readily  available.  The  point  is  that  personnel  characteristics,  along  with  unit  training,  can 
be  statistically  related  to  readiness  and  to  unit  performance  in  order  to  evaluate  the  effect  of 
MPT  policies  on  performance  via  their  effect  on  the  characteristics  of  personnel.  MPT 
policies  also  can  be  linked  to  the  kinds  and  amounts  of  resources  expended.  These  links 
are  shown  in  Figure  2-4,  which  extends  the  relationships  described  in  Figure  2-3. 

The  policy  instruments  shown  to  the  left  in  Figure  2-4  are  in  most  cases 
straightforward.  Recruiting  practices,  pay  and  allowances,  retirement  programs,  rotation 
and  quality  of  life  policies  influence  who  enlists  and  who  is  retained.  Trainers,  schools, 
rotation,  fuel  and  spare  parts  affect  operational  and  non-operational  training,  and 
experience.  Fuel,  spare  parts  and  equipment  affect  supply  and  material  readiness  directly. 
At  least  some  of  these  factors  can  be  expected  tc  affect  motivation  and  unit  personnel 
stability. 

Of  course,  these  policy  instruments  do  not  work  independently  of  the  national 
environment  any  more  than  they  do  of  unit  policies.  Other  things  equal,  fewer  recruits  will 
be  attracted  when  entry-age  cohorts  are  small,  and  more  will  be  enlisted  and  retained  when 
civilian  job  opportunities  are  poor.  Such  external  factors  should  have  (and  have  had)  a 
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prominent  place  in  MPT  policy  analysis.  However,  the  preponderance  of  MPT  research 
has  stopped  short  of  extending  the  analyses  beyond  personnel  characteristics  and  individual 
training.  Instead,  it  is  generally  assumed  that  intelligence,  education,  training,  personnel 
stability  and  experience  are  good,  and  more  is  better.  This  is  true,  but  incomplete.  Our 
primary  interest  is  in  the  portion  of  the  figure  to  the  right  of  the  policy  instruments.  We 
assume  that  certain  personnel  characteristics  and  individual  training  are  obtainable  at  some 
price.  The  question  is,  what  is  their  value  in  terms  of  military  capability? 

E.  APPLICABILITY  OF  THE  FRAMEWORK 

Figure  2-4  suggests  a  methodology  for  evaluating  MPT  policies  in  broader  terms 
than  are  typically  considered.4  While  not  all  of  the  links  hypothesized  in  Figure  2-4  have 
been  quantified  or  substantiated  in  general,  many  of  them  have  in  particular  cases.  Our 
purpose  is  to  show  that  the  personnel  characteristics  and  training  variables  can  be 
empirically  linked  to  unit  performance.  It  would  be  desirable  to  trace  the  postulated 
relationships  from  resources,  policies  and  personnel  characteristics  all  the  way  through  the 
various  readiness  measures  to  performance.  This  can  be  done  in  two  ways:  by  quantifying 
all  the  intermediate  links  implied  by  Figure  2-4,  or  bv  relating  resources,  policies  and 
characteristics  directly  to  indicators  of  performance. 

Accomplishment  cf  our  objective  requires  three  things:  an  ability  to  develop 
quantitative  indicators  of  individual  and  unit  performance,  an  ability  to  match  personnel 
characteristics  and  training  with  performance,  and  the  resolve  and  resources  to  do  the 
research  to  quantify  the  hypothesized  links.  Before  embarking  on  any  research  effort  it  is 
important  to  establish  that  adequate  data  exist  to  support  the  effort.  The  next  chapter  lays 
out  data  requirements  and  reports  our  findings  concerning  the  availability  of  such  data. 


The  framework  focuses  on  the  MPT  area.  It  could  be  expanded  to  address  the  contributions  of  all  kinds 
of  resources  to  military  performance.  This  would  facilitate  cost  effectiveness  analyses  that  cut  across 
the  pillars  of  defense  capability  of  the  kind  performed  in  S.  Scott  Sutton,  et.  al.,  A  Study  of  Aviation 
Resources  and  Readiness  Relationships,  INS  Study  32,  Center  for  Naval  Analyses,  Alexandria, 
Virginia,  June  1970. 


m.  SURVEY  OF  EXISTING  DATA  BASES 


An  analysis  of  the  relationships  between  personnel  characteristics  and  individual 
and  unit  performance  requires  accepted  yardsticks  to  measure  performance.  This 
requirement  does  much  to  frustrate  performance-oriented  policy  making.  For  the  most 
part,  policy  makers  do  not  have  routine  access  to  data  on  how  well  people  do.  The 
problem  is  not  that  the  information  does  not  exist,  but  that  it  is  not  generally  available  for 
policy  analyses.  It  is  collected  for  the  purpose  of  decision  making  at  disparate  levels  within 
the  Services  and  in  support  of  different  objectives.  Additionally,  performance  data  are 
often  considered  privileged  and  dissemination  may  be  tightly  controlled. 

The  purpose  of  this  chapter  is  to  show  that  relevant  data,  including  performance 
information,  exist  for  a  wide  range  of  units,  missions  and  tasks.  Further,  the  chapter  will 
demonstrate  that  these  data  often  are  used  to  evaluate  the  performance  of  people  and  units  in 
an  operational  setting.  While  ihe  scope  of  this  study  precludes  an  exhaustive  treatment,  an 
effort  has  been  made  to  be  as  comprehensive  as  possible.  In  putting  together  a  sample  of 
data  bases,  we  have  concentrated  on  meeting  the  following  criteria: 

•  Completeness 

-  Availability  of  information  on  both  personnel  characteristics  and  performance 

-  Coverage,  in  terms  of  the  number  of  warfare  areas  within  a  Service,  echelons 

within  warfare  area  and  time  span  treated 

•  Objectivity  of  performance  data 

•  Usability 

-  Degree  and  type  of  computerization 

-  Potential  accessibility  for  research. 

The  data  bases  can  be  characterized  in  several  different  ways.  Tables  A-l  through 
A-5,  in  Appendix  A,  list  them  by  Service,  locations  (i.e.,  who  holds  them),  format 
(automated  or  hard  copy),  warfare  area,  coverage,  type  of  data  (personnel  characteristics, 
training  or  performance),  objectivity,  and  relation  to  other  data  bases.  Significant 
characteristics  not  included  in  the  tables  or  needing  amplification  will  be  highlighted  in  the 
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discussion  to  follow.  Availability  for  research  purposes  is  not  included  as  a  category  in 
the  tables.  In  all  cases  the  Service  contacts  made  available  the  name  of  the  agency  to 
contact  for  access,  and  indicated  the  absence  of  a  stated  policy  to  withhold  data  which  is 
suitable  for  research  purposes.  It  is  conjectured  that  the  availability  of  data  can  only  be 
determined  on  a  case-by-case  basis,  contingent  on  specific  research  objectives. 

In  order  to  accomplish  the  type  of  analyses  that  we  have  addressed  in  this  paper, 
three  basic  data  types  are  required:  (1)  personnel  characteristics,  (2)  individual  and  unit 
training  and  (3)  performance.  Availability  of  data  of  the  three  types  will  be  discussed  in 
turn,  with  the  most  attention  being  paid  to  performance. 

Two  principal  categorizations  of  performance  data  are  used.  One  distinguishes 
between  maintenance  performance  and  operational  performance.  The  former  focuses  on 
whether  equipment  is  working,  the  latter  on  how  effectively  it  is  used.  The  second 
categorization  addresses  the  level  to  which  performance  is  aggregated.  Available 
performance  data  fall  into  three  categories:  that  pertaining  to  individuals,  performance  at  the 
crew,  squad  or  system  level,  and  information  concerning  the  performance  of  command 
units  (a  command  unit  is  an  entity  such  as  an  aviation  squadron,  ship  or  battalion;  these  are 
generally  units  that  can  fight  independently  and  that  maintain  detailed  personnel  and  training 
records). 

A.  PERSONNEL  CHARACTERISTICS 

Personnel  characteristics  include  name,  Social  Security  Number  (SSN),  percentile 
scores  on  the  Armed  Forces  Qualification  Test  (AFQT),  education  level,  age,  race,  ethnic 
group,  sex,  grade,  time  in  grade,  time  in  service,  occupation  code,  and  special 
qualifications.  These  data  are  maintained  in  a  comprehensive,  automated  data  base  by  the 
Defense  Manpower  Data  Center  (DMDC).  Time  in  unit  is  not  included,  but  can  be 
constructed  by  tracing  back  through  successive  records.  The  primary  DMDC  data  base  of 
interest  is  the  Enlisted/Officer  Master  File,  which  covers  all  DoD  military  personnel.  Time 
coverage  is  at  six  month  intervals  from  June  1971  to  June  1975  and  quarterly  thereafter. 
This  data  base  also  identifies  the  command  to  which  each  person  is  assigned.5  This  allows 
the  records  to  be  isolated  for  every  individual  in  a  given  unit  at  the  beginning  of  each 
quarter.  By  sorting  on  occupation,  it  is  also  possible  to  approximate  the  personnel 


Unit  Identification  Code  (UIC)  for  Army  and  Navy,  Reporting  Unit  Code  (RUC)  for  USMC,  Personnel 
Account  Symbol  (PAS)  for  USAF. 
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structure  by  weapons  system  for  some  units;  however,  in  general  it  is  necessary  to  go  to 
individual  units  (ship,  squadron,  battalion)  to  get  the  SSN  information  necessary  to  allow 
accurate  characterization  of  individual  and  aggregate  personnel  characteristics  by  crew  or 
other  sub-UIC  element 

B.  TRAINING  DATA  BASES 

Training  data  exist  for  every  person  in  the  Service,  but  unit-administered  training 
information  is  frequently  maintained  only  at  the  unit  level.  Each  Service  maintains 
automated  personnel  data  files  at  their  Headquarters  Military  Personnel  Center.  These  data 
bases,  which  are  the  source  for  the  DMDC  files,  include  occupation  and  special 
qualifications  codes  in  some  form. 

With  the  exception  of  the  Army,  records  of  attendance  of  basic  and  advanced 
individual  training  (Service  schools)  are  maintained  in  these  central  files.  The  Navy  tracks 
every  trainee  through  each  school  attended,  whether  or  not  the  course  was  successfully 
completed.  Table  A-l  lists  the  primary  centralized  and  automated  training  data  bases  for  the 
four  Services. 

Local  training  generally  is  not  available  above  the  level  at  which  the  Service  record 
is  kept  (except  for  some  flight  training).  This  is  the  battalion  level  for  Army,  ship/squadron 
level  for  Navy,  battalion/squadron  level  for  the  Marine  Corps,  and  base  level  (squadron  for 
some  training)  for  Air  Force. 

In  some  cases,  training  data  bases  include  information  on  performance. 

I .  Army 

a.  Enlisted  Master  File  (EMF)/Officer  Master  File  (OMF) 

An  automated  version  of  each  enlisted  and  officer  service  record  is  maintained  at  the 
Military  Personnel  Center,  Personnel  Information  Directorate.  However,  not  all  the 
information  kept  in  the  official  record  is  forwarded  to  MILPERSCEN  for  inclusion  in  the 
EMF/OMF.  The  OMF  contains  information  on  schools  attended  and  records  of 
assignment.  The  EMF  contains  Additional  Skill  Identifiers  (ASIs)  and  Special 
Qualification  Codes  (SQCs),  but  does  not  show  information  about  the  school  where  the 
qualification  was  acquired.  Grades  for  schools  attended  are  not  included  in  either  of  these 
files,  nor  are  records  of  on-the-job  training  (OJT)  or  annual  qualifications,  such  as 
weapons  firing  scores. 


b  Training  Management  Control  System  (TMACS) 

TMACS  is  an  automated  training  reporting  system  in  place  worldwide  and  used  at 
the  battalion/brigade  level.  The  system  is  used  to  report  individual,  crew  and  collective 
training.  TMACS  Training  Event  Codes  (TECs)  are  aligned  with  those  used  by  the  Army's 
system  of  top-down  resource  allocation  models  and  programs.  These  include:6 

•  Standards  in  Training  Commission  (STRAC), 

•  Hying  Hour  Program  (FHP),  and 

•  Operating  Tempo  (OPTEMPO). 

The  Army  Readiness  Training  Evaluation  Program  (ARTEP),  described  in  more 
detail  in  the  performance  section,  states  performance  standards  for  particular  tasks  and 
conditions  (environment).  The  purpose  of  STRAC  is  to  concurrently  specify  training 
strategies  and  annual  battalion  training  resources  required  for  each  individual,  crew  and  unit 
to  attain  and  sustain  the  ARTEP  standards.  STRAC  allocates,  in  addition  to  ordnance  and 
range  time,  simulators  and  devices  such  as  small  bore  inserts.  The  Hying  Hour  Program  is 
used  to  specify  the  quantities  of  flying  hours,  spares,  and  repair  parts  needed  to  meet 
training  requirements  at  the  individual,  crew,  unit  and  combined  arms  levels.  OPTEMPO 
establishes  requirements  for  operating  tempo,  and  actual  operating  tempo  is  recorded  in 
terms  of  vehicle  miles  and  vehicle  hours  per  time  period. 

Exercises  such  as  live  firing  are  periodically  conducted  for  qualification.  The 
scores  can  be  compared  with  quantitative  measures  of  training  resources  expended  by  the 
unit,  as  well  as  Army  standards  for  resource  usage.  TMACS  data  therefore  include  both 
training  and  performance  (as  well  as  control)  variables.  The  data  are  available  at  the 
battalion  level  as  part  of  the  TMACS  data  base. 

2.  Navy 

a.  Enlisted  Master  Record  (EMR)/Officer  Master  Record  (OMR) 

The  EMR/OMR  files  are  essentially  automated  reproductions  of  the  Navy  Service 
Record,  and  are  maintained  by  the  Deputy  Chief  of  Naval  Operations  (Manpower, 
Personnel  &  Training)  (OP-012G).  Assignment  histories,  including  schools  attended,  are 
kept  in  these  files,  as  is  information  on  civilian  education;  unit  training  and  OJT  are  not 

6  These  programs  are  discussed  in  some  detail  in  "Linking  Peacetime  Training  Resources  to  Wartime 
Combat  Readiness,"  by  Colonel  Jack  A.,  Pellicci,  Proceedings  of  the  Symposium  on  the  Military 
Value  and  Cost-Effectiveness  of  Training,  Brussels,  Belgium,  7-9  January  1985. 
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included.  These  files  are  a  quarterly  snapshot  of  each  person's  official  record  and  serve  as 
the  Navy  source  for  the  DMDC  Eniisted/Officer  Master  Files. 

b.  Student  Master  File  (SMF) 

The  SMF  is  generated  by  the  Office  of  the  Chief  of  Naval  Training  as  part  of  the 
Navy  Integrated  Training  &  Resources  Subsystem  (NITRS).  The  SMF  tracks  all 
individuals  (officer  and  enlisted)  who  attend  a  Navy  school-when  they  started,  length  of 
attendance  and  whether  they  passed  or  failed.  It  does  not  include  grades.  The  same 
information  is  also  summarized  by  course  in  the  Training  Summary  File.  The  information 
contained  in  the  SMF  is  forwarded  quarterly  to  the  Center  for  Naval  Analyses  (CNA). 
CNA  processes  the  enlisted  data  in  order  to  make  it  more  useful  for  research  purposes  and 
stores  it  in  an  automated  file. 

c.  Naval  Flight  Information  Reporting  System  (NAVFLIRS) 

NAVFLIRS  is  an  integrated  aviation  flight  hour  reporting  system  which  was 
implemented  on  January  1, 1987.  The  system  combines,  in  one  automated  data  base,  flight 
hour,  logistics  and  maintenance  information  pit  viousiy  reported  separately.  Information 
contained  in  NAVFLIRS  includes  completion  of  flights  and  simulator  periods  that  are  part 
of  the  training  syllabus,  night/instrument  hours,  approaches  and  landings.  Provision  is 
made  for  recording  completed  ordnance  training  exercises,  type  of  ordnance  expended  and 
scores:  however.  Navy  units  are  not  required  to  report  this  information.  The  data  base  is 
cumulative,  so  in  addition  to  being  able  to  obtain  information  on  specific  flights,  data 
reflecting  individual  crew  experience  can  be  derived. 

d.  Maintenance  Training  Improvement  Program  (MTIP) 

MT1P  is  an  aviation  training  program  which  contains  information  on  the  proficiency 
of  individual  maintenance  personnel.  The  MTIP  data  base  uses  the  Aviation  Training 
Support  System  (A  TSS)  computer  network,  which  is  widely  accessible.  All  aviation  type- 
wings  on  the  east  coast  have  access  to  ATSS  and  a1!  have  an  MTIP  program.  The  program 
is  built  around  a  data  hank  of  questions  which  are  sorted  by  aircraft  type,  system,  and 
Navy  Enlisted  Classification  Code  (NEC;.  Written  tests  are  given  to  all  maintenance 
personnel  periodically.  W1  an  weaknesses  show  up,  refresher  courses  are  organized  ?nd 
taught  by  Technical  Representatives  and  Naval  Aviation  Maintenance  Training  Detachment 
(NAMTD)  instructors.  The  data  base  can  be  sorted  by  squadron,  wing,  type  cf  aircraft, 
system,  etc.  Data  for  an  individual  include  time  since  last  course  attendance,  last  test  score, 
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relative  standing  within  squadron,  wing,  etc.  The  ATSS  data  base  also  includes  school 
information  other  than  the  MTIP  courses,  as  well  as  other  personnel  characteristics. 

e.  Type  Commander  Ship  Employment  Schedules 

Operating  tempo  (the  number  of  days  at  sea)  is  one  measure  of  ship  training 
accomplishment.  Type  Commanders  issue  detailed  quarterly  schedules  for  all  the  ships 
under  their  operational  control.  These  schedules  can  be  used  to  reconstruct  operating 
tempo,  using  a  system  of  over  400  distinct  "employment  terms"  to  describe  the  daily  ship 
operations.  These  "employment  terms"  indicate  training  exercises  conducted  in  port  as  well 
as  at  sea.  At  the  end  of  each  quarter,  the  schedules,  marked  up  to  show  any  deviations,  are 
returned  to  the  fleet  commands,  who  forward  them  to  the  Total  Force  Fleet  Operations  and 
Readiness  Division  (OP-64)  for  retention.7 

3.  Marine  Corps 

a.  Manpower  Management  System  File 

Officer  and  enlisted  master  files  are  arranged  by  SSN  and  Reporting  Unit  Code 

(RUC).  The  file  is  a  quarterly  snapshot  maintained  at  Headquarters  Marine  Corps.  The 

master  automated  service  record  files  are  maintained  in  Kansas  City  and  are  updated 

electronically  by  diary  entries  sent  from  the  field.  Current  and  the  previous  two 

assignments  are  shown  with  reporting  and  detachment  dates.  Up  to  eight  schools  are 

shown  with  dates  of  completion.  Grades  are  not  included  in  school  information.  The 

complete  service  record  is  maintained  at  the  battalion/aviation  squadron  level,  as  is 

information  on  most  OJT. 

* 

b.  Flight  Readiness  Data  System  (FREDS) 

FREDS  was  the  forerunner  of  NAVFLIRS  and  is  primarily  of  interest  as  a  source 
of  historical  data.  The  USMC  is  currently  a  part  of  the  NAVFLIRS,  under  a  common 
management  and  data  processing  system.  Marine  units  are  required  to  report  ordnance 
delivery  performance  data,  as  of  January  1987. 

c.  Maintenance,  Analysis,  Training  Management  and  Evaluation 
Program  (MATMEP) 


Linda  C.  Cavrlluzzo,  "OPTEMPO  and  Training  Effectiveness, "Proceedings  of  the  Symposium  on  the 
Military  Value  and  Cost  Effectiveness  of  Training.  Brussels,  Belgium,  7-9  January  1985. 
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MATMEP  is  the  USMC  equivalent  of  MTlP.  The  Marines  are  currently  in  the 
process  of  acquiring  ATSS  terminals  and  the  system  will  be  fully  integrated  with  MTIP. 
Plans  include  expanding  the  system  so  that  it  can  be  used  to  track  and  coordinate  all 
maintenance  training  and  personnel  resource  allocation. 

4.  Air  Force 
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a.  Master  Personnel  File  (MPF) 

An  automated  service  recoM  file  which  covers  both  enlisted  personnel  and  officers 
is  located  at  the  Air  Force  Military  Personnel  Center  (AFMPC),  Randolph  AFB,  Texas. 
The  file  contains  assignment  histories  which  include  Service  schools.  Provision  is  made 
for  the  inclusion  of  grades,  but  this  is  not  always  filled  in.  Air  Force  Specialty  Codes 
(AFSC),  including  prefixes  and  suffixes  and  the  date  the  codes  were  acquired,  are 
included;  AFSC  prefixes  and  suffixes  indicate  special  skills  and  special  equipment 
qualifications,  respectively.  P\S  codes  (unit  identification  codes)  and  SSNs  also  are 
included  in  the  file. 

b .  HORIS  Report 

The  HORIS  data  base  is  a  centralized  automated  data  base  which  is  maintained  at 
Air  Staff  Headquarters,  Warrior  Management  (XOOTW).  Current  experience  and  training 
data  are  maintained  for  all  active  duty  aircrew  personnel.  Information  includes  experience 
(total  and  by  aircraft  type)  for  greatest  amount  of  time  and  two  most  recent  aircraft,  combat 
time,  daily  flight  time  and  sorties  in  the  last  30  days,  etc.  The  information  can  be  sorted  on 
any  one  of  eighty  data  fields.  These  data  were  used  along  with  bombing  scores  in  a  recent 
USAF  analysis  of  the  effect  of  experience  (total,  type,  mission)  and  number  of  evolutions 
on  bombing  performance.8 

c.  Air  Force  Operations  Resource  Management  Systems  (AFORMS) 

AFORMS  is  a  standardized  reporting  and  data  base  system  for  training  information. 
Its  use  is  not  required  by  all  commands.  The  Tactical  Air  Command  made  it  mandatory  for 
wings  in  1986.  AFORMS  is  the  input  mechanism  for  the  HORIS  report.  The  data  are 
more  detailed  than  those  summarized  in  the  HORIS  report,  and  where  AFORMS  is  used  it 


Lt  Col  Thomas  E.  Cedel  and  Lt  Col  Ronald  P.  Fuchs,  "An  Analysis  of  Factors  Affecting  Pilot 
Proficiency,"  USAF  Air  Staff  Analysis  Branch,  Washington,  D.C.,  December  1986. 
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is  a  source  of  very  detailed  aviation  training  information.  Additionally,  many  standardized 
AF  forms,  such  as  the  Personnel  Training  Form,  are  entered  into  the  AFORMS  data  base. 

C.  PERFORMANCE  DATA  BASES 

In  terms  of  the  objectives  of  this  study  and  the  taxonomy  developed  in  Chapter  II, 
existing  performance  indicators  sometimes  approximate  measures  of  unit  capability;  often 
as  not  they  provide  information  more  consistent  with  unit  readiness,  training  readiness, 
material  readiness  or  personnel  readiness.  In  no  case  are  they  perfect.  Their  flaws  should 
be  borne  in  mind,  but  the  decision  whether  or  not  to  use  them  should  depend  not  only  on 
their  imperfections,  but  also  on  the  alternatives:  the  commonly  used  measures  such  as 
entry  test  scores,  education,  experience  and  pay  grade  mix,  and  operational  activity  that 
have  a  clearer  tie  to  cost  than  to  capability.  The  desire  for  perfection  should  not  be  allowed 
to  precl  ide  the  possibility  of  substantial  improvement 

1 .  Army 

a.  Army  Readiness  Training  Evaluation  Program  (ARTEP) 

ARTEP  in  itself  is  not  a  data  base,  however,  it  is  the  basis  for  essentially  all  Army 
unit  training  and  performance  evaluations.  Data  bases  are  created  in  complying  with  the 
reporting  requirements  of  ARTEP,  and  in  using  and  validating  the  programs  and  models 
that  support  it 

ARTEP  provides  a  list  of  mission  elements  and  sub-elements  that  each  kind  of  unit 
(crew/squad  through  battalion)  should  be  able  to  accomplish,  along  with  standards  for  their 
accomplishment  These  mission  elements  and  standards  are  used  by  the  unit  commander  to 
formulate  unit  training  plans,  and  to  track  and  evaluate  the  unit's  performance  and  training 
icadiness.  Both  internal  and  external  evaluations  are  used  as  diagnostic  tools  for  planning 
and  executing  unit  training. 

Internal  and  external  ARTEP  evaluations,  along  with  individual  and  crew 
qualification  scores,  can  be  combined  with  training  data  from  the  Army's  Enlisted  and 
Officer  Master  Files  (EMF/OMF),  and  personnel  data  available  from  either  DMDC  or 
Enlisted  Records.  The  performance  scores  may  not  yet  be  available  in  automated  form;  the 
Army  is  procuring  personal  computers,  but  their  use  is  not  standardized. 

Although  individual  and  crew  performance  data  have  been  used  extensively  for 
research  purposes,  external  ARTEP  evaluations  have  not.  Ostensibly  because  the  purpose 
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of  every  evaluated  evolution  is  to  train  and  to  qualify  the  participants,  the  Army  is  reluctant 
to  make  the  results  of  external  evaluations  readily  available  to  higher  authority.  The 
consensus  seems  to  be  that  such  use  would  degrade  the  integrity  of  the  system  and  result  in 
attempts  to  cover  up  rather  than  highlight  and  learn  from  deficiencies. 

Regardless  of  the  many  objections  to  using  ARTE?  evaluations,  they  provide  in 
many  instances  (particularly  those  evaluated  by  outside  observers)  relatively  objective 
measures  of  unit  performance.  It  is,  of  course,  essential  that  the  researcher  be  aware  of  the 
conditions  under  which  the  exercise  was  conducted  and  evaluated  (seldom,  if  ever,  are 
MPT  analysts  able  to  design  an  experiment  with  perfect  controls  and  unambiguous  cause- 
effect  relationships).  Despite  the  flaws  in  ARTEP  evaluation  data,  analysts  could 
reasonably  expect  to  use  them  to  identify  systematic  correlates  of  performance. 

b.  National  Training  Center  (NTC) 

Th'*  NTC  is  a  large  instrumented  ground  combat  range  located  at  Fort  Irwin, 
California.  Armored  and  mechanized  battalion  task  forces  train  in  live-fire  and  force-on- 
force  engagements  against  a  resident  and  well-  trained  opposing  force.  Exercises  involve 
the  combined  operations  of  tanks,  mechanized  infantry,  artillery,  air  defense,  engineers, 
electronic  warfare,  nuclear,  biological  and  chemical  warfare,  attack  helicopters,  and  close 
air  support  aircraft. 

NTC  training  consists  of  approximately  eleven  exercises,  including  offensive  and 
defensive,  day  and  night  operations  conducted  over  a  twelve  day  period.  The  exercises  are 
conducted  without  breaks  and  without  outside  support,  so  units  are  required  to  demonstrate 
full  logistics  capability,  including  maintenance,  evacuation  of  casualties,  and  field  billeting 
and  messing. 

Exercise  data  are  obtained  by  monitoring  radio  transmissions,  and  by  using  one 
mountain-top  and  eight  mobile  video  cameras;  digital  data  are  obtained  from  the  Multiple 
Integrated  Laser  System  (MILES).  Soldiers  and  vehicles  are  equipped  with  sensors  which 
register  hits  and  near  misses  by  laser  beams  which  simulate  weapons  fire.  Assessment  of 
degree  of  damage  and  casualties  inflicted  by  non-instrumented  weapons,  such  as  mines  and 
grenades,  are  made  by  126  observers  who  also  make  subjective  performance  judgments. 
The  observers  and  NTC  staff  together  conduct  debriefings  and  prepare  a  comprehensive 
take-home  package  intended  to  assist  the  exercise  units  in  correcting  deficiencies  after  they 
leave  the  NTC. 
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An  electronic  clipboard  currently  under  development  is  intended  to  improve  the 
effectiveness  of  the  observers.  The  clipboard  is  essentially  a  small  computer  which 
provides  menu-driven  information,  and  into  which  data  can  be  fed  using  an  electronic 
pencil.  This  device  will  make  check  lists  and  rules  of  engagement  available  to  the 
observers  nearly  instantaneously. 


The  NTC,  while  potentially  the  most  objective  source  of  performance  data  for  large 
ground  warfare  simulations,  has  not  been  exploited  to  build  a  comprehensive  data  base. 
The  NTC  Analysis  Division  of  the  Center  for  Army  Lessons  Learned  (CALL)  is 
responsible  for  collection  and  analysis  of  NTC  data.  CALL,  located  at  Fort  Leavenworth, 
publishes  a  monthly  "Lessons  Learned"  newsletter  based  primarily  on  NTC  observations 
and  discussions  with  the  NTC  Operations  Division,  members  of  the  Opposing  Force  and 
Unit  Commanders.  The  newsletters  examined  by  the  authors  do  not  contain  any  reference 
to  actual  events  or  units.  The  reasons  for  this  appear  to  be  both  technical  and  doctrinal.  On 
the  technical  side,  the  MILES  coverage  does  not  include  all  troops,  air  defense/aviation 
units  or  nuclear/chemical  warfare;  neither  does  MILES  always  penetrate  smoke  and  dust 
generated  on  the  battlefield,  nor  will  it  reach  units  masked  by  terrain.9  Additionally,  the 
observer/trainers  maj  actually  affect  the  outcome  cf  the  exercises  in  their  efforts  to 
stimulate  action  and  enhance  training.10  The  quality  of  the  data  is  therefore  considered 
suspect  by  some. 

On  the  doctrinal  side,  the  DoD,  in  responding  to  a  GAO  finding  that  the  "lack  of 
reliable  objective  data  precludes  Army-wide  assessments",  stated  that  the  primary  mission 
for  the  NTC  is  training.11  As  stated  above,  it  is  apparently  the  Army  position  that  too 
much  emphasis  on  objective  data  could  be  detrimental  to  that  mission. 


Despite  this,  it  is  clear  that  the  Army  is  taking  action  to  increase  the  role  of  the  NTC 
as  a  source  of  objective  data.  The  scope  of  MILES  is  being  expanded  to  include  indirect  fire 
and  air  defense/aviation  and  nuclear/chemical  play,  which  involves  new  software  and 
expanded  computer  capacity.12  Additionally,  the  Army  has  tasked  ARI  to  begin 
"...developing  objective  data  requirements,  a  methodology  for  the  use  of  NTC  findings  in 


10 

11 


12 


Discussions  with  Jack  Hiller,  Director,  Training  Research  Laboratory,  Army  Research  Institute. 

Ibid. 

General  Accounting  Office,  Army  Training,  National  Training  Center's  Potential  Has  Not  Been 
Realized,  GAO/NSIAD-86-130;  July  1986,  pp.  25-26. 

Ibid.  p.  26. 
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doctrine,  organization,  equipment,  and  training  development,  and  methcds  for  continuing 
to  improve  the  utility  and  quality  of  NTC  data." 13  As  part  of  this  research,  ARI  is  defining 
additional  unit  performance  measures  using  the  ARTEP  scoring  scale,  a  necessary  step  if 
the  NTC  is  to  realize  its  potential  as  a  source  of  objective  performance  data. 

c.  Crew/squad  Level  Performance  Data 

Crew  evaluations  are  conducted  throughout  the  Army.  Every  unit  conducts 
crew/squad  training  in  accordance  with  requirements  for  specific  qualifications.  This 
training  also  provides  many  of  the  validation  exercises  conducted  in  support  of  the 
Battalion  Level  Training  Model  (BLTM)  Program.  The  BLTM,  which  uses  STRAC  and 
OPTEMPO  standards,  is  intended  to  relate  training  activities  to  the  level  of  training 
readiness  based  on  the  frequency  of  execution  needed  to  sustain  various  levels  of  training 
readiness.14  Exercises  also  are  conducted  as  formal  studies  by  Army  Research 
Laboratories  in  order  to  update  BLTM,  OPTEMPO  and  STRAC  models  and  standards. 

Weapons  crews/squads  conduct  attainment  and  sustainment  events  throughout  the 
training  cycle.  Attainment  events  are  graded  exercises  conducted  for  qualification; 
sustainment  events  represent  day-to-day  practice  and  are  intended  to  build  and  maintain 
proficiency.  Riflemen  qualify  both  individually  and  collectively  as  part  of  a  squad. 
Weapons  crews  qualify  individually  and  collectively  as  well.  These  events  exercise  the 
crew  in  those  evolutions  necessary  for  each  required  mission.  Generally,  simulators  are 
used  to  the  maximum  extent  possible  for  sustainment,  and  attainment  events  are  live  firing 
exercises.  For  weapons  which  fire  very  expensive  ordnance  (e.g.,  TOW  missile) 
simulators  may  be  used  for  attainment.  Records  of  attainment  event  scores  are  kept  at 
battalion  headquarters,  usually  in  an  automated  file  whereas  sustainment  event  data  are  kept 
in  non-standard  hard  copy  format.  However,  a  prototype  automated  system  called  the 
Integrated  Training  Management  System  (ITMS)  exists  at  one  base.  The  ITMS  will 
provide  records  of  resources  expended  by  each  crew  as  well  as  attainment  event 
performance.  In  general  the  battalion  is  the  highest  level  at  which  complete  data  sets  exist, 
since  personnel  characteristics  and  training  data  are  also  kept  at  that  level.  However, 
information  about  some  individual  OJT  training  may  be  obtainable  only  at  the  company 
level. 


Ibid.  p.  27. 

op.cit.;  Col  Jack  A.  Pellicci,  pg.  334 


In  addition  to  regular  battalion-level  training  exercises,  the  Army  conducts  an 
extensive  research  program  aimed  at  evaluating  the  effectiveness  of  new  systems  and 
updating  the  various  resource  models  as  new  equipment  is  introduced.  One  such  study 
analyzed  the  effect  of  simulator  training  on  Ml  tank-firing  exercises.  Six  battalions 
participated.  All  but  one  battalion  conducted  a  specified  number  of  simulator  firings  in 
addition  to  a  specified  number  of  preparatory  live  firings.  Each  battalion  then  fired  (and 
was  graded)  on  the  same  range  under  approximately  the  same  conditions.  In  addition  to 
their  primary  objective,  such  studies  can  be  a  source  of  unusually  accurate  performance 
data.  The  researcher  is  usually  on  site  when  the  data  are  collected  and  may  be  able  to 
specify  what  data  are  to  be  recorded.  The  unit  commander  is  interested  in  qualifying 
crews,  not  necessarily  in  whether  they  qualify  on  the  first  attempt.  Unless  recording  rules 
are  specified  in  advance,  failures  prior  to  attaining  a  qualifying  score  may  not  be  recorded. 
Consequently,  performance  data  from  the  unit  training  data  base  may  be  difficult  to 
interpret. 

d.  Individual  Training  Evaluation  Program  (ITEP) 

ITEP  applies  to  every  soldier  in  skill  levels  1-4  (E-7  and  below),  active,  reserve  and 
National  Guard.  The  purposes  are  to: 

•  evaluate  proficiency, 

•  promote  standardization  of  individual  training, 

»  serve  as  a  basis  for  collective  training  plans, 

•  provide  training  feedback  to  commanders  and  MOS  proponents,  and 

•  provide  objective  indicators  of  MOS-task  proficiency  for  use  in  career 
planning. 

Three  methods  for  evaluating  individual  performance  are  used.  These  are  (in  order 
of  decreasing  objectivity)  the  Skill  Qualification  Test  (SQT),  Common  Task  Test  (CTT) 
and  Commanders  Evaluation.  The  SQT  is  a  written  test  designed  to  evaluate  and  compare 
soldiers  in  the  same  MOS  and  skill  level  across  the  Army.  Scores  are  maintained  in  an 
automated  data  file  at  HQDA  Military  Personnel  Center  (MILPERSCEN)  and  at  Enlisted 
Records,  Fort  Benjamin  Harrison,  Indiana.  The  CTT  is  a  hands-on  test  designed  to 
measure  combat  and  survival  skills  common  to  all  MOS's.  The  test  is  administered  to  all 
skill  level  1-4  personnel  at  least  once  each  year.  It  may  be  taken  by  E-8/E-9  personnel  and 
officers  at  their  discretion.  The  test  is  scored  pass/fail  and  is  repeated  until  passed. 
However,  the  number  of  skill  areas  (out  of  a  total  of  17)  passed  on  the  first  attempt  is 
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recorded.  Commanders'  evaluations  are  administered  internally  in  a  variety  of  training 
situations  and  cover  both  MOS-specific  and  common  skills.  Commanders'  evaluations  are 
also  made  as  spot  checks  during  external  ARTEP  evaluations. 

An  Army  wide  automated  ITEP  data  base,  the  Enlisted  Master  File,  is  maintained  at 
the  Army  Training  Support  Center,  Fort  Eustis,  Virginia.  Test  results  are  also  sent  to  the 
individual,  unit  commander,  Enlisted  Records  and  MILPERSCEN. 

2.  Navy 

a.  Surface  Training  and  Competitive  Cycles 


Training  plans  and  inspections  for  surface  ships  and  submarines  are  based  on  their 
regular  overhaul  (ROH)  and  deployment  cycles.  On  completion  of  an  overhaul,  or  Post 
Shakedown  Availability  (PSA)  for  a  new  ship,  a  ship  goes  through  a  period  of  basic 
training  and  general  orientation.  This  basic  phase  is  followed  by  Underway  Refresher 
Training  (RFT),  an  intensive  training  period  conducted  under  the  supervision  of  the  Reet 
Training  Group/Unit.  During  the  six  to  ten  weeks  of  RFT,  the  ship  is  exercised  in  every 
mission  and  system,  culminating  in  a  major  Battle  Problem.  Numerical  scores  for  these 
exercises  (called  Standard  Readiness  Requirements)  and  the  Battle  Problem  are  recorded 
and  reported  to  the  participating  ship  and  the  administrative  squadron  commander.  These 
evolutions  are  scored  using  subjective  as  well  as  objective  criteria  however,  they  are  scored 
at  approximately  the  same  point  in  the  training  cycle  and  each  ship  of  the  same  type  is 
graded  by  the  same  team  of  inspectors.  Overall  performance  scores  are  reported  to  and 
retained  by  the  administrative  squadron  commander.  Detailed  scores  for  each  evolution, 
which  reflect  sub-element  (division)  performance  for  many  evolutions,  are  retained  by  the 
Reet  Training  Group  on  hard  copy.  These  are  not  centralized  in  a  readily  retrievable  data 
base. 

Following  RFT,  each  ship  completes  scheduled  exercises  at  sea  and  in  port,  in 
preparation  for  deployment.  Training  exercises  (TRXs)  and  inspections  are  geared  to 
bringing  the  ship  to  a  high  level  of  readiness  and  maintaining  this  level  throughout  the 
deployment.  During  the  deployment  and  inter-deployment  periods  the  ship  completes 
TRXs  with  a  prescribed  frequency.  It  also  undergoes  certain  inspections  which  contribute 
to  its  ranking  for  battle  efficiency /excellence  awards. 

Superimposed  on  the  training  cycle  is  a  competitive  (award)  cycle,  usually  of  18 
months  duration.  In  addition  to  the  overall  Battle  Efficiency  "E",  given  to  the  best  ship  of 


each  ship  type  in  every  squadron,  excellence  awards  are  given  for  each  warfare  mission 
area  as  well  as  certain  non-mission  categories.  Awards  are  given  for  Anti-Submarine 
Warfare  (ASW),  Anti-Air  Warfare  (AAW),  Anti-Surface  Warfare  (ASU),  Command, 
Control  and  Communications  (CCC),  Mobility  (MOB),  Fleet  Support  (LOG/FSO), 
Electronic  Warfare  (ELW),  Mine  Warfare  (MIW),  Amphibious  Warfare  (AMW),  Supply, 
Damage  Control,  Navigation/Seamanship,  Aviation  Intermediate  Maintenance  and 
Retention.  Although  the  overall  excellence  awards  are  not  particularly  good  candidates  as 
performance  measures  because  of  the  small  number  of  ships  that  earn  them,  the  more 
broadly  distributed  mission  (and  non-mission)  area  awards  are.  In  addition,  some  of  the 
scored  evolutions-whieh  make  up  the  measures  on  which  awards  are  based-are  potential 
sources  of  objective  performance  data. 

(i)  Selected  Exercises  (SELEXs) 

Some  TRXs  are  performed  only  once  between  overhauls,  while  others  must  be 
repeated  more  often.  SELEXs  are  a  subset  of  the  TRXs  which  are  repeated  at  prescribed 
intervals  and  gra  led  by  outside  observers.  They  also  must  be  repeated  at  least  once  each 
competitive  cycle,  and  scores  entered  into  the  ship's  ranking  for  "E"  nominations. 
SELEXs  cover  all  primary  missions.  They  include  AAW  and  gunfire  support  live  firing, 
ASW  exercises,  and  damage  control  and  propulsion  plant  (mobility)  exercises.  The 
exercises  themselves  are  the  same  as  many  of  those  performed  during  Underway  Refresher 
Training  and  may  be  accomplished  during  RFT,  or  between  deployments,  during  a  shorter 
period  at  the  Fleet  Training  Group  called  an  Interim  Underway  Refresher  Training  (IRFT). 
When  accomplished  during  RFT/IRFT,  the  detailed  grade  sheets  are  retained  by  the  Fleet 
Training  Group. 

(ii)  Training  Inspections 

Reported  unit  readiness,  commanding  officer  evaluations,  and  awards  are  based  on 
various  inspections  given  throughout  the  training  and  competitive  cycles.  In  general, 
results  of  these  inspections  are  reported  to  the  administrative  squadron  commander  with 
summary  reports  going  to  the  type  commander.  Since  February  1987,  inspection  results 
and  the  date  training  exercises  are  completed  have  been  maintained  in  a  centralized 
automated  data  base.  However,  this  data  base,  called  the  Type  Commander  Headquarters 
Automated  Information  System  (THAIS),  contains  only  summary  data.  It  is  still 
necessary  to  go  to  the  administrative  commander  or  Refresher  Training  Group/Unit  for 
detailed  data. 
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•  Operational  Readiness  Evaluations  (OREs)/Operational  Readiness 
Inspections  (ORIs)  are  given  in  each  deployment  cycle.  They  are  given 
either  as  part  of  RFT/IRFT  or  by  the  administrative  squadron  commander 
during  the  inter-deployment  period.  The  total  RFT  package  is  essentially 
equivalent  to  an  ORE;  the  Battle  Problem  is  equivalent  to  an  ORI. 

•  Operational  Propulsion  Plant  Examinations  (OPPE)  examine  a  ship's 
capability  to  perform  the  mobility  mission.  The  OPPE  is  administered  by  the 
Fleet  CinC  Propulsion  Examining  Board  (PEB)  every  two  years,  either  before 
deployment  or  during  the  return  transit.  Every  other  OPPE  can  be  observed  by 
the  Fleet  Training  Group  during  RFT/IRFT  and  certified  to  the  PEB. 

•  Command  Inspections  are  given  approximately  six  months  before  each 
regular  change  of  command.  The  emphasis  is  on  administration  and  the 
material  condition  of  the  ship.  They  are  subjectively  scored  and  do  not  reflect 
operational  performance.  They  do  indicate  to  some  extent  the  management 
ability  of  the  senior  enlisted  and  officer  complement  of  the  unit 

b.  Aviation  Training  and  Competitive  Cycles 

Aviation  training  cycles  are  based  on  deployments.  Competitive  cycles  are  18 
months,  as  is  the  case  for  ships. 

(i)  Competitive  Exercises 

Aviation  units  also  perform  graded  exercises  which  enter  into  the  calculation  of  unit 
readiness  and  the  squadron's  standing  in  the  awards  competition.  These  include  ASW,  air 
to  surface  (missile,  bombing),  air-to-air  and  Airborne  Early  Warning  (AEW)  exercises. 
ASW  and  air-to-air  exercises  provide  objective  data  when  performed  on  an  instrumented 
range.  Instrumented  underwater  ranges  are  located  on  each  coast  and  Hawaii.  Air  Combat 
Maneuvering  Ranges  (ACMRs)  are  located  on  each  coast.  These  use  the  Tactical  Aircrew 
Combat  System  (TACTS),  a  highly  advanced  and  realistic  instrumentation  system. 
TACTS  provides  information,  instantaneously  and  on  tape,  on  the  flight  dynamics, 
weapons  system  status,  and  weapons  firing  of  each  aircraft  engaged  in  the  exercise. 

For  objectivity,  air-to-ground  live-fire  exercises  (including  those  where  dummy 
ordnance  is  used)  rec!  >  ire  only  that  an  observer  be  available  to  score  the  runs.  AEW 
exercises  can  be  objectively  graded  by  ships,  or  on-board  observers  in  some  aircraft. 

The  results  of  competitive  exercises  are  summarized  at  the  type  wing  level.  The 
format  is  generally  hard  copy,  although  some  squadrons  and  wings  are  beginning  to  use 
personal  computers.  Computer  data  formats  are  not  standardized.  Data  sheets  generally 
list  crews  by  name  so  these  can  be  matched  with  personnel  characteristics.  The  advent  of 
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the  NAVFLIRS  system  in  early  1987  should  make  considerable  data  on  training  experience 
centrally  available;  however,  historical  training  data  generally  are  not  complete  above  the 
squadron  level.  Certain  flight  information  is  available  from  the  Readiness  Analysis  Branch 
of  the  Office  of  the  Chief  of  Naval  Operations  (OP-515)  or  the  Safety  Center  for  previous 
fiscal  years. 

(ii)  Wing  Detachments  and  Competitions 

Carrier  air  wings  deploy  to  Fallon,  Nevada  each  training  cycle  for  intensive 
coordinated  ordnance  operations.  These  operations  are  conducted  under  simulated  combat 
conditions  and  are  a  source  of  objective  performance  data  for  crews  and  larger  flight 
elements.  Performance  data  are  maintained  at  the  air  wing  level.  However,  as  with 
squadron  training  history  exercises,  some  training  and  experience  data  are  available  only  at 
the  squadron  level. 

Competitions  at  the  wing  and  fleet  level  also  are  a  source  of  objective  data. 
However,  these  are  "top  gun"  type  competitions  in  which  only  the  best  from  each  squadron 
or  air  wing  are  represented.  Such  competitions  provide  highly  non-representative  stratified 
data. 

(iii)  Naval  Aviation  Operating  Procedures  and  Standards  (NATOPS) 
Qualifications 

Each  year  every  naval  aviator  and  flight  officer  must  requalify  in  their  primary 
model  aircraft.  The  annual  NATOPS  evaluation  consists  of  a  written  exam,  and  simulator 
and  actual  flight  checks  by  the  squadron  NATOPS  officer.  Pilots  (crews  of  multi-seat 
aircraft)  also  are  given  a  tactical  check  in  the  weapons  system  trainer  (WST).  These  flight 
checks  arc  considered  highly  objective  even  though  the  grading  is  based  on  the  judgment  of 
the  NATOPS  officer.  The  reason  is  that  the  NATOPS  officer  billet  is  a  highly  selective 
assignment  and  the  NATOPS  program  is  taken  very  seriously  by  the  entire  aviation 
community. 

(iv)  Inspections 

In  recent  years  the  naval  aviation  community  has  deemphasized  the  performance 
aspect  of  training  inspections.  The  Operational  Readiness  Evaluation  (ORE)  has  been 
replaced  by  the  Advanced  Phase  Evaluation  (APE).  The  ORE  for  a  Carrier  Battle  Group, 
for  example,  was  a  major  evaluation  of  all  units  (ships  and  air  wing)  in  a  simulated  combat 
environment,  and  was  administered  just  prior  to  deployment.  Individual  units  (ships  and 


aviation  squadrons)  were  observed  and  graded  by  the  Type  Commander  (TYCOM).  The 
results  of  these  evaluations  were  used  in  arriving  at  the  unit's  ORE  grade,  a  heavily 
weighted  factor  in  the  rankings  for  the  Battle  Efficiency  Award.  The  APE  is  similar  to  the 
ORE,  except  it  is  given  earlier  in  the  training  cycle.  Additionally,  the  emphasis  is  on 
whether  or  not  the  unit  needs  additional  workup  prior  to  deployment,  and  in  what  areas. 
Numerical  grades  are  not  given. 

Certain  quantitative  measures  can  be  derived  from  APE  operations.  Carrier 
landings  are  graded  by  the  landing  signal  officer  (LSO);  landing  scores  and  boarding  rates 
(the  percentage  of  successful  landings)  are  recorded  and  retained  by  the  type  commander. 
These  are  considered  to  be  two  of  the  best  indicators  of  individual  aviator  and  ship/wing 
combined  performance.  NATOPS  written  exams  also  are  given  to  all  aircrew  personnel, 
and  these  are  objective  indicators  of  squadron  training  performance.  Maintenance  Material 
Manager* 'nt  (3M)  data  collected  during  an  APE  also  are  particularly  good  maintenance 
performance  measures  because  the  APE  is  conducted  under  intensive  operating  conditions. 
The  3M  system  (to  be  discussed  below)  publishes  maintenance  performance  indicators  for 
both  ships  and  aircraft  These  indicators,  such  as  fully  capable  rates  and  utilization  rates, 
are  more  meaningful  when  the  unit  is  being  pushed. 

c.  AS  rv  Readiness/Effectiveness  Measurement  Programs 

Ship  .*  ti-Submarine  Warfare  Readiness/Effectiveness  Measurement  (SHAREM) 
exercises  ar.  ’  *r  Effectiveness  Measurement  (AIREM)  exercises  are  intended  to  collect 
operational  dz  'or  analyses  of  equipment  performance  and  tactics  development.  They  are 
observed  by  professionals  who  are  in  a  position  to  evaluate  the  overall  performance  of  the 
participating  units  s  well  as  sub-elements  by  system.  The  evaluations  have  elements  of 
both  objectivity  and  subjectivity.  Many  of  the  exercises  are  conducted  on  instrumented 
ranges,  howeve*  . .  is  not  always  possible  to  control  for  equipment  degradations  or 
environmental  conditions. 

SHAREM  data  are  maintained  in  a  central  automated  data  base.  A  computer  AIREM 
data  base  is  currently  being  developed.  The  concept  is  being  extended  to  battle  groups 
(BGAREM),  for  which  a  centralized  automated  data  base  is  also  being  developed. 

A  major  drawback  to  these  exercises  as  sources  of  information  on  personnel 
performance  is  the  inability  to  tie  performance  directly  to  individual  aircrews  or  the 
personnel  who  are  on  watch  when  the  exercise  is  conducted.  A  ship's  crew  is  divided  into 


watch  teams.  In  order  to  complete  a  data  set  for  a  given  exercise  it  would  be  desirable  to 
retrieve  indivi  dual  day-to-day  watch  bills  which  are  available  only  on  board  the  ship. 

d.  Simulators 

WST  flight  simulators  are  used  extensively  throughout  the  aviation  community. 
Many  of  these  are  highly  realistic,  and  all  have  the  advantage  of  allowing  the  operator  to 
replicate  environmental  conditions  and  opposing  forces.  Simulators  have  been  used  for 
research  purposes  with  apparently  excellent  results.13  They  can  be  used  for  nearly  every 
competitive  exercise  and  are  perhaps  superior  to  actual  flight  experience  for  many  AEW 
exercises. 

Ship  combat  information  center  (CIC)  and  propulsion  plant  simulators  also  are  used 
for  training  and  could  be  used  for  generating  performance  data.  Task  group  simulators  are 
located  on  each  coast  and  the  Navy  has  a  sophisticated  theatre  simulator  (called  NEWS)  at 
the  Naval  War  College  that  is  used  for  large  battle  problems.  The  NEWS  is  used  by  fleet 
commanders  for  evaluation  of  plans  and  ship/task  group  commanders  and  staffs. 

In  general,  the  observations  concerning  availability  of  personnel  characteristics  and 
training  data  apply  to  simulator  exercises  as  well  as  to  those  conducted  at  sea. 

e.  Integrated  RAINFORM  Analysis  System  (IRAS) 

Every  ASW  sortie  requires  the  sending  of  a  message  called  a  Rainbow  Purple.  This 
includes  major  exercises,  competitive  exercises  and  contacts  on  foreign  submarines.  These 
messages  are  correlated  for  each  aircraft  operating  in  a  given  exercise,  and  the  data  are 
maintained  in  a  central  computer  data  base.  The  purpose  of  the  data  base  is  to  serve  the 
research  community.  Crews  are  identified  by  name  but  not  SSN,  however,  matches  can  be 
made  based  on  UIC.  For  the  period  before  NAVFLIRS,  specific  experience  and  training 
data  must  still  be  obtained  from  squadron  records,  but  the  data  base  highlights  performance 
in  an  operational  setting.  It  therefore  is  an  excellent  starting  point  for  analysis  of  the 
determinants  of  operational  performance. 

f.  Naval  Aviation  Flight  Information  System  (NAVFLIRS) 

As  noted  earlier,  NAVFLIRS  is  a  new  program  instituted  in  January  1987.  It  is 
patterned  after  a  USMC  program  called  Flight  Readiness  Data  System  (FREDS)  and  is 


applicable  to  USMC  and  Navy  flight  crews.  NAVFLIRS  expands  the  previous  flight 
reporting  system  and  centralizes  flight,  logistics,  and  some  performance  and  training  data  in 
a  single  data  base.  In  addition  to  the  flight  hour  and  landing  information  previously 
reported,  provision  is  made  for  reporting  scores  achieved  on  bombing/missile  firings  on  the 
flight  log  (yellow  sheet).  Syllabus  flights  and  the  completion  of  competitive  exercises  are 
also  reported.  These  data  are  then  kept  in  a  central  data  base  along  with  logistics  and 
maintenance  information. 

g.  Maintenance  Data 

While  operational  performance  is  the  pointed  end  of  the  spear,  Navy  ships  and 
aircraft  arc  integrated  man-machine  systems.  In  fact,  one  of  the  problems  with  many  of  the 
performance  data  sources  is  the  difficulty  in  sorting  out  equipment  and  operator 
performance.  For  example,  once  an  aircraft  leaves  the  runway  on  a  bombing  competitive 
exercise,  a  failing  grade  due  to  an  equipment  malfunction  may  not  be  distinguishable  from 
one  caused  by  pilot  error.  Fortunately,  maintenance  data  are  extensive,  centralized  and,  in 
most  cases,  objective.  They  provide  information  on  the  performance  of  maintenance 
personnel  as  well  as  the  coordination  between  operations  and  maintenance. 

(i)  Aviation  Maintenance  and  Material  Management  Program  (3-M) 

The  most  comprehensive  maintenance  data  base  is  provided  by  the  Aviation  3-M 
system.  The  3-M  data  are  derived  from  maintenance  action  forms  generated  by  work 
centers  at  the  squadron,  intermediate  (AIMD),  and  depot  maintenance  activities. 
NAVFLIRS  data,  training  device  utilization  and  maintenance  information  also  are 
processed  and  reported  by  3-M.  The  system  is  the  regular  maintenance  reporting  and 
management  system  for  both  the  Navy  and  Marine  Corps. 

3-M  data  are  used  to  develop  reports  on  mission  capable  (MC)  and  fully  mission 
capable  (FMC)  rates  that  are  routinely  used  to  track  the  material  readiness  of  aircraft. 
Details  such  as  non-faulty  parts  removed  and  average  time  to  repair  by  individual  work 
center  and  part  number  also  can  be  monitored.  Statistics  such  as  mean  time  between  failure 
and  time  to  repair  can  be  collected  for  component  levels  down  to  individual  parts,  by 
manufacturer. 

3-M  reports  are  available  in  a  variety  of  formats  and  for  any  organizational  level  and 
type  aircraft  Data  also  are  available  on  magnetic  tape  for  specialized  research  use  with 
permission  of  the  Department  of  the  Navy.  Data  can  be  readily  correlated  with  Navy  and 


USMC  Enlisted  Master  Files,  NAVFLIRS  aircraft  flight  hours  and  flight  code  data,  and 
Maintenance  Training  Improvement  Program  data. 

(ii)  Surface  Maintenance  and  Management  Material  (3-M) 

Surface  3-M  data  are  not  as  comprehensive  nor  as  standardized  as  are  Aviation  3- 
M.  These  data  are  detailed  to  the  level  of  manufacturer  and  Equipment  Identity  Code. 
Operating-hour  data  for  ships  are  not  reported  by  3-M.  Steaming  hours,  broken  down  by 
employment  by  day,  are  available  from  the  CINCs  and  OP-643,  and  The  Visibility  and 
Management  of  Operating  and  Support  Costs  (VAMOSC)  data  base  also  contains  some 
information  about  steaming  hours. 

(iii)  Form  4855  Data 

These  data  reflect  information  on  equipment  logs  for  selected  combat  systems.  It  is 
possible  to  tell  when  the  equipment  was  on,  when  failures  were  detected,  when  parts  were 
on  order,  and  when  failures  were  corrected.  These  data,  which  are  kept  at  the  Fleet 
Analysis  Center  under  the  sponsorship  of  NAVSEA,  are  very  clean  but  not  always 
accessible. 

(iv)  Casualty  Reports  (CASREPs) 

When  a  ship  suffers  an  equipment  failure  that  adversely  affects  its  ability  to  carry 
out  its  primary  missions  (and  cannot  be  repaired  within  48  hours),  a  casualty  report  is 
made.  CASREPs  range  in  severity  from  C-2  (partial  degradation)  to  C-4  (loss  of  ability  to 
perform  at  least  one  mission).  CASREPs  are  subjective  but  they  are  taken  very  seriously  in 
determining  supply  priority,  and  by  the  type  commanders.  CASREP  data  are  readily 
available  from  the  Ships  Parts  Control  Center. 

(v)  Board  of  Inspection  and  Survey  (INSURV) 

The  INSURV  Board  inspects  approximately  100  ships  each  year,  on  average  each 
ship  is  inspected  every  six  years.  Inspectors  are  highly  trained  and  respected  for  their 
consistency.  The  INSURV  is  a  material  inspection  which  divides  the  ship  into  25 
equipment  categories.  Each  category  is  scored  according  to  a  scale  on  which  zero  indicates 
no  deficiency,  one  a  deficiency  and  two  a  mission  degradation.  The  sum  of  the  categorical 
scores,  called  a  Material  Condition  Index  (MCI)  is  then  calculated.  The  system  is  highly 
aggregated;  for  instance,  all  combat  equipment  is  included  in  one  category.  The  INSURV 
Board  has  recently  developed  a  more  detailed  data  base  for  selected  equipment. 
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3.  Marine  Corps 

a.  Marine  Corps  Combat  Readiness  Evaluation  System  (MCCRES) 

This  system  is  similar  to  the  Army  ARTEP  in  some  respects  and  is  applicable  to 
both  ground  and  aviation  units.  The  mission  of  every  kind  of  unit  is  subdivided  into  tasks, 
and  tasks  into  requirements.  The  mission  of  an  infantry  battalion,  for  example,  is  divided 
into  900  such  requirements.  Each  requirement  is  defined  such  that  its  satisfactory 
accomplishment  can  be  described  by  a  yes  or  no.  Tasks  and  requirements  are  not 
necessarily  equal  in  their  contribution  to  total  mission  and  each  is  given  a  weight  in 
determining  overall  unit  score. 

MCCRES  measures  the  performance,  under  simulated  combat  conditions,  of  each 
unit  against  a  set  of  well  defined  standards  called  Mission  Performance  Standards  (MPS). 
For  an  infantry  battalion  there  are  17  MPSs  divided  into  four  sections:  all  evolutions; 
amphibious  assault  and  normal  combat  operations;  specialized  combat  operations  and  use 
of  outside  support  assets.  Each  section  contains  between  three  and  seven  MPSs,  each 
containing  several  tasks.  Each  task  in  turn  consists  of  several  requirements.16 

An  evaluation  is  conducted  by  several  observers  who  assign  a  value  of  1  or  0  to 
each  requirement  according  to  whether  it  was  satisfactorily  executed.  Internal  evaluations 
are  conducted  throughout  the  training  cycle  to  assist  unit  commanders  in  evaluating  training 
readiness  and  planning  training  programs.  At  least  every  two  years  each  unit  is  evaluated 
by  a  command  specified  by  the  Commander,  Fleet  Marine  Force. 

Data  from  external  MCCRES  evaluations,  called  Readiness  Evaluations,  are 
maintained  at  the  inspected  unit  down  to  the  squadron/battalion  level.  The  evaluation 
records,  which  are  stored  on  PC  diskettes,  are  also  forwarded  to  HQMC. 

b.  Aviation  Training  and  Readiness  Information  Management 
System  (ATRIMS) 

ATRIMS  keeps  track  of  aircrew  training  and  performance  (officer  and  enlisted). 
Syllabus  flights  and  exercises,  for  each  pilot  and  observer,  are  entered  and  compared  with 
required  re-fly  intervals  and  scores  in  order  to  calculate  the  Combat  Readiness  Percentage 
(CRP).  CRP  is  a  measure  of  readiness  which  enters  into  the  squadron's  reported  unit 


S.  Zacks,  W.  H.  Marlow  and  S.S.  Brier,  "Statistical  Analysis  of  Very  High-Dimensional  Data  Sets  of 
Hierarchically  Structured  Binary  Variables  with  Missing  Data:  An  Application  to  Marine  Corps 
Readiness  Evaluations,"  Naval  Research  Logistics  Quarterly,  (32)  1985,  pp.  467-490. 
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readiness.  Each  aviator  has  a  certain  CRP  at  the  completion  of  flight  training  which 
increases  with  experience. 

ATRIMs  draws  on  the  inputs  to  the  FREDS/NAVFLIRS  data  bases.  However, 
ATRlMs  cannot  be  accessed  by  any  echelon  above  the  squadron  level.  In  general  all 
personnel  data,  exercise  scores  (air  to  ground,  air  to  air,  AEW,  etc.),  Standardization  Flight 
scores,  and  carrier  landing  scores  for  sea-based  squadrons  are  available  at  the  squadron 
level. 

c.  Maintenance  Performance  Data 

The  Aviation  3-M  system  was  described  under  Navy  Maintenance  Performance 
Data.  The  Marine  system  is  essentially  the  same  and  is  maintained  by  the  same 
organization.  The  ground  maintenance  data  base  is  called  the  Marine  Corps  Integrated 
Maintenance  Management  System  (MIMMS).  MIMMS  is  neither  as  centralized  nor  as 
extensive  as  3-M,  primarily  because  it  deals  with  less  complex  systems.  Inputs  are  initiated 
by  Equipment  Repair  Orders,  which  identify  a  piece  of  equipment  that  is  down,  the  specific 
system,  and  required  maintenance  and  supply  actions.  These  orders  are  forwarded  to  one 
of  four  world-wide  processing  centers  where  they  are  retained  for  IS  months.  Daily  and 
monthly  reports  summarize  the  information  by  equipment,  but  not  by  unit 

4.  Air  Force 

Within  the  Air  Force,  performance  data  bases  reside  primarily  at  the  wing  level; 
detailed  exercise  grades  may  exist  only  at  the  squadron  level.  Inspections,  daily  training 
and  graded  readiness  exercises,  and  maintenance  performance  are  reported  to  the  wing, 
major  command  (MAJCOM)  and  Department  of  the  Air  Force  at  decreasing  levels  of  detail. 
Specific  reports,  reporting  procedures  and  formats  may  vary  among  MAJCOMS.  The 
following  discussion  is  based  primarily  on  detailed  interviews  with  Tactical  Air  Command 
personnel. 

a.  Aircrew  Training  Program 

Unit  training  requirements  are  set  by  the  MAJCOM,  which  is  responsible  for 
providing  necessary  resources  for  accomplishing  the  training.  Squadrons  are  responsible 
for  conducting  training  evolutions  and  qualifying  aircrews  in  certain  events,  initially  and  at 
prescribed  intervals,  according  to  the  experience  of  each  individual.  Training  cycles  are  six 
months,  beginning  the  first  of  January  and  July.  The  system  of  flight  hour/syllabus 
training  requirements  is  called  the  Graduated  Combat  Capability  System.  Different 
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numbers  of  training  flights/hours  are  required  during  each  cycle  for  experienced  and 
inexperienced  aircrews.  Several  kinds  of  performance  information  are  generated  as  a  result 
of  the  aircrew  training  program. 

(i)  Weapons  Qualification  Scores 

Crews  must  qualify  in  certain  weapons  events  to  be  certified  as  Mission  Ready 
(MR).  Qualifying  events  also  include  EW  and  tanking  and  logistics  events  for  the  Strategic 
Air  Command  and  Military  Air  Transport  Command  (SAC/MAC).  Detailed  score  sheets 
(AF  forms  206  and  107)  for  weapons  qualification  events  are  retained  by  the  squadron  for 
at  least  the  current  and  previous  training  cycles  (they  may  be  retained  in  individual  training 
records  for  a  longer  period,  but  this  is  not  standardized).  Gun-camera  film  is  used  for 
scoring  and  is  perhaps  the  most  objective  source  of  weapons  event  performance  data; 
however,  there  is  no  requirement  to  retain  film  for  any  specific  length  of  time  and  retention 
varies  across  units. 

SAC  scores  for  both  daily  training  and  qualification  events  afe  maintained  by  the 
First  Combat  Evaluation  Group.  For  events  flown  prior  to  July  1987  it  is  not  possible  to 
match  sorties  with  individual  aircrews  without  going  to  the  squadron.  However,  the  form 
(1CEVG  95)  contains  space  for  this  information,  and  commencing  in  July  1987  the 
primary  aircrews  last  four  SSN  digits  and  first  two  letters  of  the  last  name  are  included  on 
the  form. 

Weapons  qualification  event  scores  are  generally  a  source  of  objective  performance 
data.  These  data  were  used  along  with  HORIS  data  in  the  previously  cited  USAF  analysis 
of  the  effect  of  experience  on  bombing  performance.17 

(ii)  Wing  Detachments  and  Competitions 

TAC  wings  deploy  to  Nellis  AFB,  Nevada  on  an  18  month  cycle,  where  all 
aircrews  who  have  progressed  to  a  specified  point  in  the  syllabus  participate  in  exercises 
called  Red  Flag  (Green  Flag  for  ECM).  The  Air  Force  Human  Resources  Laboratory 
(AFHRL)  is  currently  analyzing  Red  Flag  data  along  with  personnel  and  training  data 
obtained  from  DMDC  and  AFMPC.18 


op.cit.,  Lt  Col  Thomas  E.  Cedel,  and  Lt  Col  Ronald  P.  Fuchs. 
Discussions  with  N.  Greenhalgh,  AFHRL. 


Annual  "Top  Gun"  competitions  are  held  at  the  squadron  level  through  the 
MAJCOM  level.  These  culminate  in  competitions  (William  Tell,  air-to-air:  Gunsmoke,  air- 
to-ground)  among  the  top  teams  from  each  MAJCOM  within  the  Tactical  Air  Forces.  SAC 
also  has  annual  bombing/SRAM  competitions.  As  noted  previously.  Top  Gun  awards  are 
highly  selective,  and  provide  non-representative  data.  However,  interesting  results 
concerning  the  effect  of  equipment  sophistication  across  delivery  profiles  have  been 
obtained  using  Gunsmoke  data.19 

(iii)  Standardization/Evaluation  (STAN/EVAL) 

Every  aircrewman  must  pass  an  annual  comprehensive  evaluation  of  overall 
airmanship  and  operational  ability,  which  is  administered  by  the  wing  Standardization 
Board/Evaluator.  Records  of  flight/simulator  checks  and  written  exams,  ACMR  results 
and  gun-camera  film  are  kept  at  the  wing  and  squadron  levels.  With  the  exception  of 
SAC,  the  wing  STAN/EVAL  program  is  evaluated  by  the  MAJCOM  Inspector  General 
during  Operational  Readiness  Inspections  (ORIs).  The  First  Combat  Evaluations  Group 
conducts  ORI  evaluations  of  SAC  STAN/EVAL  programs  and  maintains  the  data. 

b .  Inspections 

Operational  and  Administrative  inspections  are  conducted  by  the  MAJCOM 
Inspector  General  (IG).  Wings  also  conduct  Self-Initiated  Operational  Readiness 
Inspections  (SIORIs)  in  preparation  for  the  IG  inspection.  The  format  is  essentially  the 
same,  with  all  events  being  scored  by  the  wing. 

(i)  Operational  Readiness  Inspection  (ORI) 

The  wing  is  notified  that  an  ORI  will  be  given  sometime  within  a  45  day  period,  but 
is  not  told  the  exact  date.  Phase  I  of  the  ORI  evaluates  the  wing's  initial  response  and  its 
ability  to  deploy  the  wing  assets.  Phase  II  tests  its  ability  to  plan  and  execute  operations  in 
a  simulated  combat  environment.  Simulated  combat  sorties  are  flown  against  an  opposing 
force  for  approximately  a  four  day  period.  Because  of  the  duration  of  the  exercise,  supply 
and  maintenance  are  not  tested  for  sustainability.  Adherence  to  doctrine  and  24-hour 
aircraft  availability  during  the  four  day  period  are  evaluated  by  professional  teams  of 
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K.  Lindsey,  Lt  Col  Paul  R.  Dordal,  and  Lt  Col  M.S.  Brake,  Gunsmoke  1985  Statistical  Summary  and 
Lessons  Learned,  USAF,  HQ  TAC,  Directorate  of  Operations  Analysis,  Tech  Report  86-2,  March 
1986. 
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observers.  The  exercises  are  designed  to  evaluate  the  wing  in  every  weapon-delivery 
profile. 

Gun-camera  film  and  ACMR  data  are  evaluated,  and  these  are  retained  at  the  wing 
along  with  score  sheets.  Personnel  characteristics,  training,  and  performance  data  are 
therefore  all  available  at  the  wing. 

(ii)  Management  Effectiveness  Inspection  (MEI) 

The  MEI  is  an  administrative  inspection  of  the  entire  base,  and  concentrates  on 
administration,  special  programs  and  the  support  side  of  the  base,  wing  and  squadron. 
Grading  is  detailed  and  systematized,  but  subjective.  The  grading  criteria  reflect  adherence 
to  doctrine;  however,  the  observer  teams  are  professional,  and  every  base  within  a 
MAJCOM  is  observed  by  the  same  team. 

c .  Simulators 

Either  4  or  6  sorties  are  required  each  half  cycle  in  Aircrew  Training  Devices 
(ATDs),  depending  on  experience.  ATDs  are  also  used  in  specialized  training  programs 
and  for  Stan/Evals  and  instrument  checks.  Detailed  score  sheets  (Form  206s)  are  filled  out 
for  ATD  sorties  and  retained  at  squadron  or  wing  level  as  appropriate. 

d.  Maintenance  Data 

The  maintenance  data  reporting  system  resides  at  the  wing  level  with  summary 
reports  going  to  the  MAJCOM.  The  system  is  similar  to  the  Navy  3-M  system. 
Maintenance  status  is  kept  by  aircraft  and  maintenance  squadron,  and  summarized  every  24 
hours  in  the  Daily  Status,  Flying  arid  Performance  Report.  Data  which  are  summarized 
daily  include  mission-capable  rates,  full- mission-capable  rates,  utilization  rates,  aborts  (air 
and  ground),  repeats,  sorties,  break  rates,  fix  rates  and  cannibalization  rates.  Additional 
data  summarized  monthly  at  the  MAJCOM  level  include  Foreign  Object  Damage  (FOD)  and 
dropped  object  damage. 

Maintenance  Training  Information  is  also  maintained  at  the  wing  level  as  part  of  the 
centralized  aircraft  maintenance  system.  An  OJT  and  required  task  book  is  kept  on  each 
person.  Individual  progress  and  qualifications  are  tracked  in  order  to  maintain  certification. 


IV.  SUMMARY  AND  RECOMMENDATIONS 
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A.  SUMMARY 

Conceptual  links  exist  between  MPT  policy  and  Defense  Department  outputs  — 
military  performance,  readiness  and  defense  capability.  The  use  of  explicit  quantitative 
links  to  such  outputs  could  lead  to  the  development  of  improved  MPT  policy. 

The  lack  of  performance  data  traditionally  has  been  given  as  the  reason  for  not 
developing  quantitative  relationships  between  MPT  policy  and  military  performance,  but 
we  have  shown  in  Chapter  III  that  performance  data  exist,  and  that  they  are  accessible  and 
in  usable  form.  Furthermore,  we  have  shown  that  information  concerning  personnel 
characteristics  and  training,  which  are  needed  to  draw  inferences  about  the  determinants  of 
unit  performance,  also  are  available.  Information  on  the  operational  and  maintenance 
performance  of  units,  crews  and  individuals  is  gathered  routinely.  Indicators  of  unit-level 
operational  proficiency  are  not  generally  available  from  any  central  repository,  but 
maintenance  and  some  individual/crew  performance  data  are  centrally  available. 

Many  sources  of  information  on  performance  in  the  military  were  identified  in 
Chapter  ID.  In  conjunction  with  available  data  on  training  and  personnel  characteristics, 
they  appear  to  be  suitable  for  output-oriented  analyses  of  personnel  and  training  policies. 
As  was  noted  earlier,  available  performance  data  faU  into  three  categories:  data  pertaining  to 
individuals,  data  at  the  level  of  crews,  squads  or  systems,  and  data  concerning  the 
performance  of  command  units.  This  section  reviews  our  findings  about  each  of  these 
three  categories  of  data  and  about  maintenance  data,  which  may  be  available  at  either  the 
system  or  command  level. 

In  general,  the  larger  the  entity  the  more  difficult  it  is  to  obtain  objective  data  on 
performance,  and  the  more  difficult  it  is  to  control  for  external  effects  and  draw  clear 
inferences.  Of  course,  the  larger  the  unit  the  more  relevant  are  any  inferences  to  our 
objective  —  understanding  the  determinants  of  military  effectiveness. 
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1.  Unit  Level  Performance  Data 


Command  unit  performance  data  --  what  we  are  most  interested  in  --  are  available  in 
some  form  for  all  four  Services.  These  are  also  the  least  used  for  analytic  purposes.  The 
reasons  most  often  given  for  failure  to  use  these  data  are: 

•  Too  much  emphasis  on  performance  could  negate  the  purpose  for  which  unit 
performance  measures  are  defined  in  the  first  place.  Allowing  access  to 
information  on  unit  standings  could  provide  an  incentive  to  cover  up  rather 
than  highlight  and  correct  deficiencies. 

•  Scoring  is  inexact,  incomplete  or  too  subjective. 

•  It  is  too  difficult  to  control  for  environmental  factors,  such  as  weather,  quality 
of  opposing  forces,  underwater  sound  profiles,  etc. 

These  reasons  are  also  used  to  explain  the  dearth  of  analyses  of  performance  at  the 
crew/squad  and  individual  levels. 

The  question  appears  to  be  not  so  much  whether  data  are  available  as  whether  the 
potential  gain  is  worth  the  cost.  The  objection  to  gathering  and  using  performance 
indicators  for  analyses  and  central  management  appears  to  be  confined  to  operational 
performance.  Information  on  maintenance  performance  is  used  routinely  by  the  Services  to 
develop,  assess  and  justify  policy  (though  more  for  policy  regarding  spare  parts  than 
personnel).  There  is  concern,  though,  that  making  data  on  operational  performance  more 
widely  available  would  dilute  the  incentives  of  commanders  in  the  field  to  evaluate  their 
subordinate  units  accurately,  for  fear  that  bad  performance  would  reflect  badly  on  them. 
However,  analyses  of  MPT  policies,  using  performance  data,  can  be  done  without  making 
the  performance  data  widely  available.  The  identity  of  the  units  that  lie  behind  the  data  can 
be  suppressed  without  damaging  the  analyses. 

The  question  of  whether  or  not  the  data  are  accurate  enough  for  use  as  a  basis  for 
MPT  policy  decisions  is  an  important  one.  However,  the  data  are  being  used  for  policy 
decisions  now.  For  example,  if  it  is  determined  that  a  unit  needs  more  training  in  a 
particular  area  before  deploying  outside  CONUS,  that  is  a  policy  decision.  If  data  are 
adequate  to  support  such  a  decision,  it  is  reasonable  to  believe  they  are  adequate  to  support 
decisions  about  specific  Service  schools,  OPTEMPO,  or  the  mix  of  grades  and  occupations 
required  for  a  deploying  unit. 

Controlling  for  environmental  factors  is  primarily  technical  in  nature.  It  is  the  job 
of  .the  analyst  to  observe  and  select  appropriate  control  variables  from  what  exists.  In 


addition,  failure  to  control  for  environmental  factors  is  unlikely  to  contaminate  the  kinds  of 
statistical  analyses  we  envision  unless  there  are  systematic  relationships  between  the 
omitted  factors  and  the  policy-related  variables  being  studied.  It  is,  for  example,  not  likely 
that  tank  crew  members  with  high  entry  test  scores  perform  graded  exercises  in  weather 
that  is  signficantly  different  on  average  from  those  with  lower  test  scores.  Unless  such  a 
difference  exists,  there  is  no  need  to  control  for  weather  in  an  analysis  of  the  importance  of 
having  high  mental  group  personnel  in  tank  crews. 

Unit  performance  can  be  measured  directly  or  aggregated  from  the  performance  of 
individuals  or  crews/systems/squads.  But  direct  measurement  is  preferable  because  it 
measures  command  performance  unambiguously.  Direct  measurement  data  also  are  more 
difficult  to  obtain.  Both  the  Army  and  Marine  Corps  have  comprehensive  standards  for 
unit  performance  and  programs  for  implementing  these  standards.  The  National  Training 
Center  provides  an  instrumented  range  for  evaluating  unit  performance  with  respect  to  these 
standards.  A  concentrated  effort  is  currently  underway  to  improve  the  quality  of  NTC 
evaluations  and  to  establish  a  quantitative  data  base.  Underway  Refresher  Training  and  the 
SHAREM  data  base  are  sources  of  direct  measurement  ship  performance  data.  Battle 
problems  and  ship  mission  exercises  (ASW,  AAU,  ASU,  etc.)  are  truly  unit  evolutions. 
Although  the  success  of  an  ASW  exercise  hinges  critically  on  the  individual  sonar 
technician,  the  ability  to  communicate,  maneuver  and  bring  weapons  to  bear  is  just  as 
crucial  to  the  success  of  the  operation.  Aviation  units  can  be  evaluated  directly  in  some 
cases.  Air  Force  ORIs  and  Navy/USMC  wing  exercises  at  Fallon,  Yuma  and  Oceana 
include  simulated  combat  exercises  involving  many  aircraft  which  test  the  coordination  of 
the  entire  unit.  AIREM  exercises  lasting  for  an  extended  time  test  the  ability  of  a  squadron 
or  wing  to  put  units  on  station,  and  effectively  operate  for  an  extended  period.  Some 
aspects  of  unit  performance  are  directly  measurable  during  APEs.  Boarding  and  launch 
rates,  for  example,  reflect  unit  performance  directly.  Standardization/Evaluation  and 
NATOPS  evaluations  also  measure  unit  performance  directly.  Aviation  units  can  also  be 
evaluated  operationally  in  two  stages.  Maintenance  performance  during  major  wing 
evaluations  measures  a  unit's  capability  to  bring  forces  to  bear,  and  individual  crew/flight 
performance  measures  its  ability  to  execute  its  mission.  The  AIREM  and  Integrated 
RAINFORM  Analysis  System  data  bases  provide  centrally  available  performance 
information  for  coordinated  flight  operations  as  well  as  for  individual  ASW  crews. 

Training  and  personnel  characteristics'  data  bases  exist  at  the  command  unit  level 
for  all  Services.  Additionally,  much  of  the  required  data  are  centralized  at  the  respective 
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Service  personnel  centers.  The  Navy  and  Marines  also  are  establishing  aviation 
maintenance  training  data  bases  as  part  of  the  Maintenance  Training  Improvement  Program 
(MTIP).  These  can  be  accessed  centrally  and  contain  essentially  all  required  training  data  at 
the  unit  level. 

2.  Crew/Squad/System  Performance  Data 

These  data  are  available  and  are  in  general  more  objective  than  command  unit 
performance  data,  but  their  link  to  ultimate  military  effectiveness  is  somewhat  mere  remote. 
For  one  thing,  there  is  an  aggregation  problem  in  building  up  to  unit  performance.  For 
example,  the  connection  between  total  unit  performance  and  performance  at  the  crew  level 
may  be  weakened  if  leadership  is  not  properly  factored  into  the  performance  model.  Better 
crews  will  perform  better  as  a  unit  ceteris  paribus,  but  the  unit's  performance  will  also 
depend  on  how  well  it  is  integrated  and  led.  Perhaps  information  on  the  characteristics  of 
unit  commanders  could  fruitfully  be  included  as  variables  in  unit  level  analyses. 

In  general,  personnel  characteristics  and  training  data  paralleling  crew/squad/system 
performance  data  reside  at  the  squadron/ship/battalion  level.  However,  for  Navy,  Marine 
and  Air  Force  aircrews  much  of  this  information  is  centralized  in  the  Marine  Corps 
FREDS,  Navy/Marine  NAVFLIRS  and  Air  Force  HORIS  data  bases.  For  all  Services, 
externally  graded  exercises  (Navy  SELEXs  and  Competitive  Exercises,  Army  qualifying 
weapons  tables.  Air  Force  CGG  and  weapons  qualification  sorties,  Marine  Corps 
ATRIMS/CRP  qualification  sorties),  or  key  checks  such  as  NATOPS  or  STAN/EVALS, 
are  the  most  promising  sources  of  crew  performance  data.  Detailed  score  sheets  as  well  as 
training  data  for  many  of  these  exercises  are  kept  at  the  wing  or  administrative  command 
level. 

Simulator  exercises  are  also  potentially  valuable.  Flight  and  gun/missile  simulators 
are  becoming  increasingly  realistic.  They  have  the  advantage  of  allowing  replication  of 
carefully  chosen  environmental  conditions. 

3 .  Individual  Performance  Data 

Throughout  our  research  and  discussions  with  personnel  we  concentrated  on 
investigating  command  units  and  crew/squad/system  data,  though  some  individual 
performance  data  bases  were  discussed.  Programs,  such  as  the  Army  Individual  Training 
Evaluation  Program  and  the  Navy/Marine  Corps  Maintenance  Training  Improvement 
Program  are  sources  of  individual  performance  data.  Such  data  can  also  be  aggregated  to 
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yield  measures  of  unit  training  performance.  Individual  performance  is  also  reflected  in 
achievement  on  Navy  advancement  exams,  or  the  Air  Force  Weighted  Promotion  System 
tests.  Advancement  history  within  an  occupational  specialty  also  can  be  used  as  a  proxy 
for  individual  performance. 

As  with  crew/squad/system  performance  measures,  individual  performance 
measures  can  be  used  to  derive  either  dependent  variables  in  studies  of  recruiting,  retention, 
promotion,  or  training  policies,  or  in  some  cases,  independent  variables  in  analyses  of  unit 
performance. 

4.  Maintenance  Performance  Data 

Maintenance  and  supply  data  are  available  at  all  levels,  and  are  generally  more 
objective  and  complete  than  operational  performance  data.  These  data  also  are  centrally 
located  in  automated  data  bases.  Maintenance  performance  directly  reflects  the  proficiency 
of  maintenance  personnel.  In  addition  to  its  important  direct  effect  on  operational 
capability,  maintenance  performance  has  an  indirect  impact  through  its  effect  on  the 
affordability  of  weapons  systems.  Approximately  seventy  percent  of  the  life  cycle  cost  of  a 
weapons  system  is  in  Operations  and  Maintenance.  Analyses  of  the  MPT  determinants  of 
maintenance  performance  could  therefore  have  a  high  payoff. 

B.  RECOMMENDATIONS 

Our  review  of  sources  of  data  on  performance  in  the  military,  and  our  acquired 
familiarity  with  the  existing  literature  that  relates  the  characteristics  and  training  of 
personnel  to  performance,  leads  us  to  four  broad  recommendations.  They  are  listed  here  in 
order  of  the  speed  with  which  they  can  be  implemented: 

•  Strive  to  use  the  results  of  existing  research. 

•  Identify  and  perform  a  set  of  initial  analyses  that  address  policy  issues  of 
particular  interest 

•  Further  institutionalize  the  development  of  automated  data  bases  that  can  be 
used  in  analyses  of  personnel  and  performance. 

•  Encourage  a  long-term  effort  for  quantifying  the  links  between  manpower, 
personnel  and  training  policies  and  measures  of  performance. 

The  remainder  of  this  chapter  briefly  discusses  these  four  recommendations. 
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1.  Using  Existing  Research 

A  modest  body  of  literature  ties  personnel  characteristics  and  training  histories  to 
measures  of  operational  or  maintenance  performance.  This  literature  could  serve  two 
fruitful  purposes.  First,  it  could  be  used  to  develop  a  set  of  performance-based  insights 
into  personnel  and  training  policies  which  might  include  the  following: 

•  A  package  of  occupation-specific  retention  bonuses  designed  to  increase 
seniority  in  the  enlisted  force.  This  would  take  advantage  of  the  finding  that, 
in  many  roles,  senior  personnel  are  substantially  more  productive  than  junior 
ones.20 

•  Plans  to  obtain  more  years  of  flying  from  pilots.  This  would  involve  cutting 
down  on  the  non-flying  assignments  of  relatively  young  pilots  and,  perhaps, 
providing  more  flying  assignments  for  older  pilots.  The  previously  cited  work 
of  Cedel  and  Fuchs  points  in  this  direction. 

•  Reconsideration  of  Navy  plans  to  provide  a  larger  fraction  of  initial  skill 
training  aboard  ship.  Such  a  policy  has  been  found  both  to  delay  competence 
and  to  distract  senior  personnel  from  their  other  duties..21 

In  addition,  existing  research  could  help  refine  efforts  to  quantify  the  overall  quality 
of  the  enlisted  military  force.  This  would  involve  using  quantitative  estimates  of  the 
relative  contribution  to  performance  of  personnel  with  different  levels  of  experience, 
intelligence  and  education  to  develop  indices  of  personnel  quality.  In  some  cases  it  would 
be  possible  to  take  into  account  that  as  a  particular  beneficial  characteristic  (like  experience) 
becomes  more  prevalent  in  the  fo^  %  its  additional  contribution  to  improved  performance  is 
likely  to  diminish.  The  work  o;  iviarcus  probably  provides  the  best  evidence  of  the  value  of 
experience  beyond  the  first  term,  while  that  of  Scribner  et  al  provides  information  on  the 
value  of  intelligence.22  Such  an  approach  would  require  one  to  assume  that  results 
obtained  for  a  small  group  of  occupations  could  be  extrapolated  broadly,  but  it  would  at 
least  provide  a  useful  check  on  (and  supplement  to)  estimates  of  enlisted  force  quality  that 
have  been  developed  to  date  —  one  more  firmly  based  on  the  performance  of  units. 


20  See,  for  example,  Deborah  C».iy-Mende.  Ellen  Balis,  Kurt  A.  Driscoll  and  Robert  F.  Lcckman, 
Balancing  Accession  and  Retention,  CNS  1176,  Center  for  Naval  Analyses,  September  1982  and  Alan 
J.  Marcus,  Personnel  Substitution  and  Naval  Aviation  Readiness,  Professional  Paper  363,  Center  for 
Naval  Analyses,  October  1982. 

2 1  See  Rodney  Weiher  and  Stanley  A.  Horowitz,  1  ~  Relative  Costs  of  Formal  and  On-the-Job  Training 
for  Navy  Enlisted  Occupations,  Professional  P"  .  J,  Center  for  Naval  Analyses,  November  1971. 

22  Barry  L.  Scribner,  D.  Alton  Smith,  Robert  H.  Baldwin,  and  Robert  W.  Phillips,  "Are  Smart  Tankers 
Better  Tankers:  AFQT  and  Military  Proficiency,  "Office  of  Economic  and  Manpower  Analysis,  U.  S. 
Military  Academy,  December  1984. 


2.  Performing  Initial  Studies 


Re-orienting  MPT  policymaking  to  focus  on  the  implications  of  policy  decisions  for 
military  performance  will  take  a  long  time,  but  the  success  of  any  long-term  effort  requires 
short-term  evidence  that  the  effort  will  be  worth  the  trouble.  The  best  possible  evidence 
would  be  the  generation  of  usable  policy  insights.  We  recommend  striving  to  generate 
such  insights  within  two  years. 

This  program  of  short-term  research  should  commence  by  deciding  what  policy 
choices  are  most  in  need  of  calibration  in  terms  of  their  impact  on  performance.  This 
should  determine  what  to  look  at  first.  Candidates  for  the  list  of  initial  studies  should 
include  the  following* 

•  The  impact  on  operational  performance  of  reduced  operating  tempo  (and, 
hence,  reduced  levels  of  unit  training)  in  all  the  Services.  Congressional 
pressure  on  this  portion  of  the  budget  makes  operating  tempo  a  timely  issue. 

•  Greater  examination  of  the  importance  of  intelligence.  Recent  military  pay 
raises  have  failed  to  keep  pace  with  civilian  wage  levels.  Evidence  is  that  this 
is  likely  to  adversely  affect  both  the  accession  and  retention  of  personnel  in  the 
upper  mental  groups.  Information  regarding  the  implications  of  this  for 
performance  could  be  most  valuable. 

•  Comparisons  of  the  proficiency  of  forces  manned  by  reserve  personnel  with 
those  manned  by  active  personnel.  There  have  been  prominent  discussions  of 
removing  some  units  from  Europe  and  converting  them  to  CONUS-based 
reserves  in  order  to  save  money.  Consideration  of  this  option  shouldn't  be 
done  without  examining  the  performance  implications  of  the  shift 

•  The  value  of  seniority.  Congress  has  noted  that  the  average  level  of  seniority 
in  the  Services  has  grown,  raising  personnel  costs.  Some  are  considering 
relying  on  existing  analyses  of  the  increased  proficiency  of  a  more  senior  force 
to  recommend  a  decrease  in  the  number  of  military  personnel  on  active  duty. 
This  may  be  appropriate,  but  further  analysis  would  help  guide  specific 
decisions  about  whether  or  not  to  cut,  how  much  to  cut  and  where  to  cut. 

•  The  impact  of  personnel  turnover  on  unit  performance.  Personnel  movement 
costs  the  military  a  great  deal;  it  also  disrupts  smooth  working  relationships 
that  have  been  built  up  over  time.  Quantifying  the  implications  of  these 
disruptions  for  performance  could  provide  an  impetus  toward  increased 
personnel  stability. 

For  all  of  these  study  areas  the  initial  step  will  have  to  be  illustrative,  since  the  same 
relationship  cannot  be  expected  to  hold  for  all  occupations  or  kinds  of  units.  For  example. 


there  is  no  reason  to  believe  that  operating  tempo  has  the  same  impact  on  proficiency  in  B- 
52s,  C-130s  and  F-15i.  But  illustrative  results  can  be  very  useful.  Finding  a  specific 
quantitative  relationship  between  operating  tempo  and  aircrew  proficiency  for  a  particular 
kind  of  aircraft  performing  a  particular  mission  is  concrete  evidence  that  readiness  suffers 
when  the  flying  hour  budget  is  cut,  even  though  we  don't  know  how  much  it  suffers  in 
most  cases.  It  shifts  some  of  tne  burden  of  proof  onto  those  who  want  to  reduce  flying 
hours.  This  is  the  kind  of  payoff  a  carefully  chosen  set  of  initial  studies  can  provide. 

3.  Developing  Computerized  Data  Bases 

As  has  been  noted,  allowing  access  to  information  on  performance  for  research 
purposes,  especially  the  performance  of  units,  is  viewed  with  considerable  skepticism  by 
the  Services.  Such  information  has  the  potential  of  revolutionizing  the  ways  in  which 
personnel  and  training  policies  are  formed,  evaluated  and  justified,  but  the  revolution  can 
only  take  place  if  performance  data  are  easily  accessible  for  analytic  purposes.  Easy 
accessibility  implies  not  having  to  start  every  study  by  convincing  suspicious  operators  that 
study  results  cannot  possibly  be  used  against  them  and  not  having  to  create  a  performance 
data  base  by  laboriously  copying  large  volumes  of  data  from  poorly  kept  paper  files. 

Can  the  legitimate  concerns  of  the  Services  be  reconciled  within  the  requirements  of 
policy-oriented  research?  We  think  that  the  only  feasible  approach  to  reconciliation  is  the 
recognition  of  the  control  of  the  Services  over  their  own  data.  If  this  control  is  recognized, 
it  should  be  possible  to  develop  data  sets  that  minimize  the  chances  for  misuse  or  punitive 
use  without  adversely  affecting  the  prospects  for  productive  research.  The  following  steps 
could  facilitate  such  development 

a.  Enunciate  a  DoD  policy  supporting  the  use  of  information  on  the  performance 
of  individuals  and  units  to  study  the  effectiveness  of  policies  regarding 
personnel  and  training.  Make  it  clear  that  the  Services  are  in  charge  of 
determining  the  nature  of  this  use. 

b .  Initiate  a  multi-Service  group  to  develop  guidelines  for  centralizing  the  location 
of  performance  data  for  each  Service.  These  guidelines  might  well  include 
removal  of  unit  and  individual  identifying  codes;  if  these  codes  were  removed, 
it  would  be  necessary  to  merge  performance  data  with  relevant  information  on 
personnel  characteristics  and  training  histories  before  the  removal  took  place. 
The  adoption  of  uniform  guidelines  would  permit  the  performance  of  parallel 
research  across  the  Services. 


c.  Request  that  the  Services  identify  the  types  of  data  to  be  computerized  and 
centralized. 

d.  Provide  funds  earmarked  for  the  purposes  of  developing  the  necessary 
software  and  doing  the  processing  required  to  develop  the  desired  data  bases. 

e.  Make  provisions  for  the  periodic  updating  of  the  data  bases. 

f .  Recognize  Service  responsibility  for  permitting  access  to  the  data  bases  for 
research  purposes.  The  data  bases  themselves  could  either  be  held  by  the 
Services  or  by  a  centralized  organization.  The  key  point  is  that  the  Services 
would  have  control  over  their  dissemination. 

It  is  possible  that  following  this  plan  would  yield  an  environment  that  still  inhibits 
the  sorts  of  policy  research  we  are  interested  in  promoting,  but  it  should  assuage  many  of 
the  concerns  about  the  misuse  of  data.  In  any  case,  without  strong  assurances  of  both 
central  support  and  Service  control,  it  seems  unlikely  that  concerted  progress  will  be  made 
toward  the  goal  of  getting  access  to  the  data  necessary  for  tying  manpower,  personnel  and 
training  policies  to  performance. 

4.  Encouraging  a  Long-term  Research  Effort 

It  is  tempting  to  suggest  a  co-ordinated  effort  to  quantify  all  the  links  in  Figures  2-3 
and  2-4,  but  this  probably,  would  be  misguided.  To  a  considerable  extent  the  policy 
development  process  is  a  decentralized  one.  The  Services,  and  particular  offices  in  the 
Services,  have  a  lot  of  independence  in  developing  many  aspects  of  MPT  policy.  They 
must  have  control  over  their  own  research  programs  in  those  areas  where  they  develop 
policy.  Still,  there  are  several  ways  in  which  the  Office  of  the  Secretary  of  Defense  can 
encourage  appropriate  research  to  link  MPT  policies  to  readiness  and  performance. 

•  Direct  that  MPT  policies  be  developed  and  evaluated  on  the  basis  of  their  effect 
on  performance,  as  much  as  possible. 

•  Review  specific  Service  policies  with  an  eye  toward  their  justification  in  terms 
of  performance.  Regularly  ask  the  Services  to  provide  such  justification. 

•  In  areas  where  a  uniform  DoD  policy  is  required,  such  as  the  level  of  pay  or 
the  nature  of  the  retirement  system,  OSD  should  sponsor  research  to  develop 
relationships  between  policy  decisions  and  indicators  of  performance. 

Taking  these  three  steps  should  produce  an  environment  which,  over  time,  will 
assure  the  quantification  of  a  wide  range  of  relationships  between  MPT  policies  and 


military  performance.  The  data  and  techniques  exist  to  build  these  relationships.  They  can 
provide  a  more  firm  underpinning  for  DoD's  personnel-related  policies  than  has  ever 
existed. 
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-  Anti-air  Warfare 

-  Air  Combat  Maneuvering  Range 

-  Airborne  Early  Warning 

-  Air  Force  Human  Resources  Laboratory 

-  Air  Force  Manpower  Center 

-  Air  Force  Operations  Resource  Management  Systems 

-  Armed  Forces  Qualification  Test 

-  Air  Force  Speciality  Code 

-  Aviation  Intermediate  Maintenance  Department 

-  Air  Effectiveness  Measurement 

-  Amphibious  Warfare 

-  Advanced  Phase  Evaluation 

-  Army  Research  Institute 

-  Army  Readiness  Training  Evaluation  Program 

-  Assistant  Secretary  of  Defense  for  Reserve  Affairs 

-  Additional  Skill  Identifier 

-  Anti-Surface  Warfare 

-  Anti-Submarine  Warfare 

-  Aircrew  Training  Devices 

•  Aviation  Training  and  Readiness  Information  Management  System 

-  Aviation  Training  Support  System 

-  Battle  Group  Air  Effectiveness  Measurement 

-  Battalion  Level  Training  Model 

-  Center  for  Army  Lessons  Learned 

-  Casualty  Reports 

-  Congressional  Budget  Office 

-  Command,  Control  and  Communication 

-  Commander-in-Chief 

-  Center  for  Naval  Analyses 
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CRP 

-  Combat  Readiness  Percentage 

CTT 

-  Common  Task  Test 

DMDC 

•  Defense  Manpower  Data  Center 

DoD 

•  Department  of  Defense 

(D)TPDC 

-  (Defense)  Training  and  Personnel  Data  Center 

ELW 

-  Electronic  Warfare 

EMF 

-  Enlisted  Master  File 

EMR/OMR 

-  Enlisted  Master  Record/ Officer  Master  Record 

FHP 

-  Flying-Hour  Program 

FMC 

-  Fully  Mission  Capable 

POD 

-  Foreign  Object  Damage 

FREDS 

•  Flight  Readiness  Data  System 

FSO 

-  Fleet  Support  Operations 

GAO 

-  Government  Accounting  Office 

HORIS 

-  Hormats'  Information  System 

HQDA 

-  Headquarters  Department  of  Army 

IG 

-  Inspector  General 

INSURV 

-  Board  of  Inspection  and  Survey 

IRAS 

-  Integrated  RAINFORM  Analysis  System 

IRFT 

-  Interim  Underway  Refresher  Training 

ITEP 

•  Individual  Training  Evaluation  Program 

ITMS 

-  Integrated  Training  Management  System 

JSPD 

-  Joint  Services  Planning  Document 

LOG/FSO 

-  Logistics/Fleet  Support  Operations 

LSO 

-  Landing  Signal  Officer 

MAJCOM 

-  Major  Command 

MATMEP 

-  Maintenance,  Analysis,  Training  Management  and  Evaluation 
Program 

MC 

-  Mission  Capable 

MCCRES 

-  Marine  Corps  Combat  Readiness  Evaluation  System 

M Cl 

-  Material  Condition  Index 

MEL 

-  Management  Effectiveness  Inspection 

MILES 

-  Multiple  Integrated  Laser  System 

MILPERSCEN 

-  Military  Personnel  Center 

MIMMS 

-  Marine  Corps  Integrated  Maintenance  Management  System 
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MIW 

-  Mine  Warfare 

MOB 

-  Mobility 

MOS 

-  Military  Occupation  Specialty 

MPT 

-  Manpower  Personnel  and  Training 

MTIP 

-  Maintenance  Training  Improvement  Program 

3M 

-  Maintenance,  Material,  Management 

NAMTD 

-  Naval  Aviation  Maintenance  Training  Detachment 

NATOPS 

-  Naval  Aviation  Operating  Procedures  and  Standards 

NAVFLIRS 

-  Naval  Flight  Information  Reporting  System 

NEC 

-  Navy  Enlisted  Classification  Code 

NITRS 

-  Navy  Integrated  Training  Resources  Subsystem 

NTC 

-  National  Training  Center 

OJT 

-  On-the-Job-Training 

OMF 

-  Officer  Master  File 

OPPE 

-  Operational  Propulsion  Plant  Examination 

OPTEMPO 

-  Operating  Tempo 

OPTIP 

-  Operations  Training  Improvement  Program 

ORE 

-  Operational  Readiness  Evaluation 

ORI 

-  Operational  Readiness  Inspection 

OSD 

-  Office  of  the  Secretary  of  Defense 

PA&E 

-  Program  Analysis  and  Evaluation 

PAS 

-  Personnel  Account  Symbol 

PEB 

-  Propulsion  Examining  Board 

PSA 

-  Post  Shakedown  Availability 

RFT 

-  Underway  Refresher  Training 

ROH 

-  Regular  Overhaul 

RUC 

-  Reporting  Unit  Code 

SAC 

-  Strategic  Air  Command 

SELEX 

-  Selected  Exercise 

SECDEF 

-  Secretary  of  Defense 

SHAREM 

-  Ship  Anti-Submarine  Warfare  Readiness/Effectiveness 
Measurement 

SIORI 

-  Self-Initiated  Operational  Readiness  Inspections 

SMF 

-  Student  Master  File 

SQC 

-  Special  Qualification  Code 

SQT 

•  Skill  Qualification  Test 

SSN 

-  Social  Security  Number 

STRAC 

-  Standards  in  Training  Commission 

TAC 

-  Tactical  Air  Command 

TACTS 

-  Tactical  Aircrew  Combat  System 

TEC 

-  Training  Event  Code 

THAIS 

-  Type  Commander  Headquarters  Automated  Information  System 

TMACS 

-  Training  Management  Control  System 

TRX 

-  Training  Exercises 

TYCOM 

-  Type  Commander 

UIC 

-  Unit  Identification  Code 

VAMOSC 

-  Visibility  and  Management  of  Operating  Support  Costs 

WST 

-  Weapons  System  Trainers 
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AND  PERFORMANCE  DATA 


TABLE  A-1.  CLASSIFICATION  OF  PERSONNEL  AND  TRAMINQ  DATA  (ALL  SERVICES) 


TABLE  A-l.  (Coni.)  CLASSIFICATION  OF  PERSONNEL  AND  TRAINING  DATA  (ALL  SERVICES) 


TABLE  A-3.  CLASSIFICATION  OF  PERFORMANCE  DATA  (NAVY) 


TABLE  A  )  (Can't.)  CLASMFtCATION  OF  PERFORMANCE  DATA 
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